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PREFACE 


This  report  covers  work  performed  during  the  period  Kay  1984 
through  January  1987  fc  :  the  Air  Force  Wright  Aeronautical 
Laboratories  Flight  Dynami  "  Laboratory  (AFWAL/FIGR)  under 
contract  F33615-63-C-3618 .  Dr.  John  M.  Reising  wr^  Project 
Manager  with  Capt.  Gretchen  Lizza,  and  later,  Lt.  James  E. 

NcClain  as  Project  Engineers.  The  authors  wish  to  thank  Dr. 
Reising,  Lt.  McClain,  and  especially,  Capt.  Lizza  for  their 
guidance  and  support  during  this  project. 

Special  acknowledgement  is  made  of  the  contributions  by  the 
Ofc,orational  Air  Force  and  Navy  flight  crews  who  evaluated  these 
formats.  These  professional  pilots  and  "guys  in  back"  served  as 
.'ary  effective  measuring  instruments  for  the  pictorial  formats. 

The  work  was  performed  by  the  Crew  Systems  Technology  Group  of 
the  Boeing  Military  Airplane  Company  in  Seattle.  J.  D.  Gilmour 
was  the  Program  Manager.  T.  C.  Way  was  the  Principle 
Investigator  and  designed  the  simulation.  Members  of  the  staff 
participated  in  all  phases  of  the  effort  but  R.  E.  Edwards  was 
primarily  responsible  for  the  test  design  and  M.  E.  Hornsby  was 
responsible  for  much  of  the  format  development.  R.  L.  Martin 
organized  the  disc  formats,  and  led  the  simulation  integration, 
checkout  and  data  collection  phases. 

The  simulation  was  conducted  in  BMAC's  Flight  Simulation 
Laboratory,  R.  A.  Becker,  Manager.  Programmers,  engineers  and 
technicians  who  worked  on  the  project  included  Craig  Betzina,  Bob 
Coyle,  Lee  Emerson,  Lou  Hough,  John  Kay,  Tom  Krogel,  Harmon  Law, 
Kevin  McMahon,  Eric  Miyamoto,  Jake  Schemnitzer,  Steve  Wagner  and 
Mike  Warden.  Their  skill  and  dedication  in  creating  and 
operating  the  simulation  contributed  strongly  to  the  program. 
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EXECUTIVE  SUMMARY 


OBJECTIVES!  A  simulator  study  was  conducted  to  evaluate  the 
usability  and  acceptability  cf  pictorial  format  displays  for  two- 
seat  fighter-attack  aircraft;  to  determine  whether  usability  and 
acceptability  were  affected  by  display  mode  —  color  or 
monochrome;  and  to  recommend  format  changes  based  on  the  results 
of  the  simulation. 

BACKGROUND:  The  missions ,  complexity,  and  capability  of  modern 
aircraft  are  approaching  the  point  where  they  are  beyond 
capability  of  aircrews  to  operate.  Integration  and  automation 
techniques  are  being  applied  to  address  this  problem.  The 
present  study  brings  to  bear  recent  advances  in  cockpit  display 
technology  which  now  allow  pictorial  representation  of  flight, 
situation,  and  airplane  system  information.  The  goal  is  to 
present  the  information  the  aircrew  needs,  when  they  need  it,  and 
in  a  form  that  is  most  useful. 

APPROACH :  Pictorial  formats  were  developed  for  a  head-up  display 
(HUD) ,  a  perspective  situation  format  (PSP),  and  a  horizontal 
situation  format  (HSP).  Two  close  look  formats  (CLPs)  were 
developed  to  show  an  expanded  view  of  aircraft  in  selected  small 
areas  during  an  air  engagement.  Additional  formats  were 
developed  to  represent  status  of  the  propulsion,  fuel, 
hydraulics,  electrical,  stores,  countermeasures,  and  passive 
sensor  systems.  Stores  programming,  countermeasures 
programming,  and  advisory  checklists  were  also  represented.  All 
these  formats  had  both  color  and  monochrome  versions. 

A  simulation  was  assembled  to  evaluate  these  formats  under 
realistic  flight  conditions.  A  two  seat  cab  was  constructed 
with  four  cathode  ray  tubes,  multi-purpose  displays  in  each  seat 
plus  a  HUD  in  the  front.  Controls  and  switches  were  added  to 
support  the  procedures  necessary  for  an  operational  misssion. 
System  malfunction,  low  level  penetration,  and  beyond-visual- 
range  air-to-air  mission  segments  together  with  full  high-low- 
high  missions  were  planned  and  created  in  digital  simulation. 

Sixteen  operation  two-man  USAF  or  USN  aircrews  each  spent  three 
days  learning  the  formats  and  the  simulated  aircraft,  then  flying 
the  missions  and  evaluating  the  monochrome  and  color  versions  of 
the  formats.  Pilot  opinion,  workload,  and  performance  data  were 
collected. 
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RESULTS  AND  CONCLUSIONS:  In  their  critiques,  the  pilots  and  WSOs 
clearly  preferred  the  color  formats.  They  indicated  general 
approval  of  the  pictorial  format  concept  and  provided  detailed 
criticism  of  specific  formats.  Subjective  workload  assessments 
did  not  show  a  significant  difference  between  color  and 
monochrome  formats.  However,  there  was  an  apparent  learning 
effect  favoring  the  color  formats.  The  performance  data  did  not 
show  significant  color/monochrome  differences.  There  were 
weaknesses  in  aircrew  performance  which  could  be  identified  with 
particular  formats. 

The  crew  critiques  and  performance  data  were  applied  to 
recommended  revisions.  The  PSF  in  air  mode  and  the  CLF  received 
the  most  extensive  revision.  Minor  changes  were  recommended  to 
other  formats. 


1 . 0  INTRODUCTION 

This  is  the  final  report  of  a  program  designed  to  further  the 
development  of  pictorial  formats  for  fighter  and  attack  aircraft. 
In  this  program,  pictorial  formats  were  evaluated  in  a  two-seat 
fighter.  The  intent  of  pictorial  formats  was  to  present 
information  in  a  native,  intuitive  way  which  minimises  mental 
processing. 

1.1  Background 

Until  quite  recently,  the  primary  medium  for  cockpit  information 
display  was  electro-mechanical  —  tapes,  gages,  flags,  and  dials 
assembled  into  indicators.  The  limited  degrees  of  freedom  in 
these  electro-mechanical  indicators  required  that  they  be 
dedicated  and  single  purpose,  that  they  show  raw  data,  and  that 
there  be  many  of  them.  There  was  a  time  when  this  was 
sufficient.  Aircraft,  aircraft  systems,  and  missions  were 
simpler. 

Since  then  however,  aircraft,  systems,  and  missions  have  all 
become  more  complex.  As  the  mission  requirements  became  more 
demanding,  technology  advanced  to  meet  those  requirements, 
providing  more  capable  aircraft  and  systems.  However,  these 
technological  advances  created  a  cockpit  information  overflow. 
Aircrews  in  modern  fighter  attack  aircraft  are  inundated  with 
information,  most  of  it  important  and  much  of  it  critical  some 
time  in  some  missions.  Ideally,  aircrews  would  be  provided  the 
information  they  need,  when  they  need  it,  and  in  a  form  which  can 
be  easily  understood  and  assimilated  into  an  overall  awareness  of 
their  current  situation. 

This  problem  is  being  attacked  on  a  number  of  fronts.  More 
efficient  uses  of  the  voice  and  auditory  channels  are  being 
explored  for  pilot  control  inputs  and  airplane  information 
outputs.  New  display  media,  including  helmet-mounted  displays, 
are  being  developed.  Advanced  sensors,  fusion  of  information, 
better  guidance  schemes,  and  advanced  automation  through 
artificial  intelligence  are  all  in  development. 

The  present  series  of  studies  is  investigating  ways  to  exploit 
the  degrees  of  freedom  available  in  electro-optical  displays. 
Specifically,  pictorial  formats  have  been  developed  to  portray 
information  needed  by  the  aircrew  in  a  native,  intuitive  manner 
to  maintain  a  clear  general  awareness  of  the  airplane  and  mission 
situation  and  specific  awareness  of  conditions  which  require 
immediate  and  reliable  aircrew  input. 
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1.2  Previous  Work 


Hi*  Multi-Crew  Pictorial  Format  Display  Evaluation  Program  it  tha 
third  in  a  aarias  of  contracted  efforts,  sponsored  prisiarily  by 
tha  Air  Force  Plight  Dynamics  Laboratory,  Craw  Systems 
Development  Branch,  (AFMAL/FIGK) .  in  the  first  of  these  efforts, 
conceptual  displays  were  developed  for  six  primary  fighter  crew 
station  functional  primary  flight,  tactical  situation,  stores 
management,  systems  status,  engine  status,  and  emergency 
procedures  (Jauer  and  Quinn,  1982). 

In  the  second  contract,  Pictorial  Format  Display  Evaluation 
(PPDt),  the  Boeing  Military  Airplane  Company  continued  the 
development  beyond  the  paper  formats  of  the  earlier  program  and 
implemented  the  results  in  a  piloted  simulation.  Two  simulation 
studies  were  conducted  to  evaluate  the  usability  and 
acceptability  of  pictorial  format  displays  for  single-seat 
fighter  aircraft;  to  determine  whether  usability  and 
acceptability  were  affected  by  display  mode  —  color  or 
monochrome;  and  to  recommend  format  changes  based  on  the 
simulations.  In  the  first  of  the  two  PFDE  studies,  pictorial 
formats  were  implemented  and  evaluated  for  flight,  tactical 
situation,  system  status,  engine  status,  stores  management,  and 
emergency  status  displays.  The  second  PFDE  study  concentrated 
on  the  depiction  of  threat  data.  The  number  of  threats  and  the 
amount  and  type  of  threat  information  were  increased.  Both  PFDE 
studies  were  reported  in  May,  Hornsby,  Gilmour,  Edwards  and 
Robbs,  1984. 

A  total  of  thirty  USAF  and  USN  pilots  in  the  two  studies  flew 
mission  simulations  with  color  and  monochrome  versions  of  the 
displays.  Objective  performance  data,,  subjective  pilot  ratings, 
and  comments  were  collected.  In  general,  the  pilots  found  the 
pictorial  format  displays,  and  the  specific  implementations  used 
in  these  studies  to  be  quite  acceptable.  They  preferred  color 
over  monochrome  versions.  A  number  of  improvements  were 
suggested  for  particular  format  elements,  and  were  incorporated 
into  revised  formats. 

1.3  Objectives 

The  present  study  had  two  primary  objectives.  One  of  these  was 
to  evaluate  usability  and  acceptability  to  two-seat  tactical  air 
crews  of  a  set  of  service-provided  pictorial  formats  for  elecvro- 
optical  displays.  The  second  objective  was  to  determine  whether 
the  degree  of  usability  and  acceptability  of  the  pictorial 
formats  was  a  function  of  two  basic  display  presentation  modes: 
monochrome  and  color.  A  further  objective  was  to  refine  the 
formats  based  on  information  gathered  during  the  simulation. 


The  fcoarw  was  intended  to  support  tha  sarvicas  in  their  af torts 
to  provide  s  firm  technology  boss  in  tha  area  oi  aircraft 
crewstatlons,  displays,  and  controls,  in  addition,  tha  work 
supports  tha  Air  Pores  Armament  and  Avionics  Laboratories  in 
thair  raspaotiva  9oals  of  developing  integrated  stores 
management  and  avionic  systems  which  are  coapa tibia  with  advanced 
craw  intarfaca  coacapts  and  workload  requirements.  These  service 
goals  are  being  pursued  through  a  nuaber  of  exploratory  and 
advanced  davalopaant  programs  that  include  tha  demonstrated 
feasibility  of  cockpit  electro-optical  displays  driven  by  high¬ 
speed  digital  computers.  The  Multi-Craw  Pictorial  Format 
Display  Program  has  furthered  these  objectives  by  simulating  and 
evaluating  a  representative  set  of  electro-optical  display 
formats  designed  to  significantly  reduce  tha  information 
processing  demands  placed  upon  flight  craws.  This  reduction  in 
mental  workload  will  allow  flight  crews  to  more  efficiently 
extract  information  from  the  cockpit. 

1.4  Organisation  of  the  Program  and  of  this  Report 

Tha  majority  of  the  formats  evaluated  hare  ware  largely  derived 
from  recommendations  at  the  end  of  the  PFDK  program.  Others  ware 
developed  locally  and  still  others  were  added  by  tha  contracting 
agency,  AFMAL/FXGR.  The  formats  ware  subjected  to  an  iterative 
development  process  with  four  evaluations. 

1.4.1  Static  Format  evaluation 

Karly,  after  the  formats  were  fairly  mature,  but  before  they  were 
committed  or  programmed,  they  were  evaluated  by  pilots  of  the 
318th  Fighter-interceptor  Squadron  stationed  at  McChord  APE .  The 
static  format  evaluation  is  described  more  fully  in  Section  2. 

1.4.2  Format  and  Simulation  Development 

Simulation  development  and  further  format  development  followed 
tha  static  format  evaluation.  Section  3  describes  the  simulation 
facilities.  Section  4  describes  the  simulated  aircraft,  its 
systems  and  the  pictorial  formats  which  support  them. 

1.4.3  Demonstration  One 

After  the  format  and  simulation  development  for  the  MCPFD  program 
were  well  underway,  it  was  decided  to  use  these  formats  and  parts 
of  this  simulation  for  a  demonstration  of  artificial 
intelligence.  The  objective  of  Demonstration  One  was  to  show 
feasibility  and  potential  of  an  expert  systems  approach  to  pilot 
decision  aiding.  This  objective  was  met  by  adding  symbolic 
processing  to  elements  of  the  MCPFD  simulation.  The  result 


highlighted  expert  systems  et  work  in  e  high  quality,  pilot-in- 
the-loop  siaulation.  The  demonstration  iteelf  wee  a  one-tiae 
event  presented  in  January  1986.  It  was  sponsored  by  the  Defense 
Advanced  Research  Projects  Agency  (DAAFA)  and  administered 
jointly  by  APWAL/PIOR  and  APNAL/AAAT.  This  siaulation  of  expert 
systeas  was  documented  in  a  full  color,  narrated,  video  tape 
(booing  Military  Airplane  Coapany,  1966a),  in  a  technical 
operating  report  (booing  Military  Airplane  Coapany,  1986b)  and  in 
a  final  report  (Pohlaann,  Shelnutt,  Stenerson,  Payne  and  Marks, 
1986). 

1.4.4  Dynaaic  Coaparisons 

After  the  foraats  and  siaulation  were  coapleted,  but  before  data 
was  collected  in  the  final  two  stages,  a  session  was  conducted  to 
select  froa  aaong  alternate  versions  of  the  dynaaic  foraats  for 
primary  flight,  tactical  situation,  and  navigation  displays. 
Inforaation  obtained  froa  this  evaluation  was  used  to  deteraine 
the  specific  versions  of  the  dynaaic  foraats  which  were  to  be 
tested  in  the  aission  segaent  and  coaposite  aission  siaulation 
evaluations.  The  dynaaic  coaparisons  stage  is  reported  in  aore 
detail  in  Section  5. 

1.4.5  Mission  Segaents  and  Coaposite  Missions 

Sixteen  two-aan  aircrews  each  participated  in  a  three-day  program 
to  learn,  use,  and  evaluate  the  foraats.  Bach  crew  flew  both 
aission  segaent  and  coaposite  aission  pictorial  format 
evaluations.  Their  program  is  detailed  in  Section  6  and  the 
results  of  their  evaluation  are  documented  in  Section  7. 

1.4.6  Conclusions  and  Revised  Foraats 

The  conclusions  and  revised  foraats  are  given  in  Section  8.  This 
is  done  as  an  application  of  performance,  opinion,  and  workload 
data  to  the  original  intention  of  each  format. 
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2.0  STATIC  FORMA?  EVALUATION 


Modifications  to  existing  formats#  and  the  development  of  now 
formats#  wars  basad  on  data  obtainad  in  aarliar  avaluation 
afforts  and  on  analysas  of  craw  rasponsibilitias  and  information 
requirements  in  a  two-saat  aircraft.  Bacausa  tha  translation  of 
display  format  concapts  into  a  functional  simulation  anvironmant 
is  a  long  and  itarativa  process,  it  was  dacidad  to  involva 
oparational  craws  in  tha  davalopmant  and  avaluation  procass  at 
tha  aarliast  possibla  stags.  Tha  purposa  of  tha  static  format 
avaluation  was  to  allow  oparational  craws  to  critiqua  proposad 
formats  and  to  usa  thair  inputs  in  furthar  modifications  of  tha 
formats  to  ba  spacifiad  for  tha  full  simulation. 

2.1  Formats 

Savantaan  format  axamplas  ;ara  submit tad  for  avaluation:  HUD, 

HUD  with  Missils  Launch  Envalopa  (MLE)  symbology#  air  and  ground 
mods  Psrspactiva  Situation  Format  (PSP)#  air  and  ground  mods 
Horisontal  situation  Format  (HSF) #  Targat  Formation#  Engine 
Status  and  Engina  Advisory#  Elactrical  Status  and  Elactrical 
Advisory#  Hydraulic  Status#  Fual  Status#  Storas  Statue# 

Counts rmaasur as  status,  and  Passiva  Sensor  status.  Two  varsions 
wara  davalopad  for  aach  format  axampla:  color  and  monochroma. 
Each  format  was  ganaratad  as  a  high  rasolution  color  or 
monochrome  transparency  using  a  computer-based  graphics  system, 
in  soma  cases  composite  axamplas  wara  used  to  illustrate  a  wider 
variety  of  display  symbology  than  would  appear  at  any  given 
instant  on  tha  cockpit  display.  A  description  was  written  for 
aach  format  axampla  to  guide  tha  evaluator  briefing  and  ensure 
that  all  important  display  symbology  was  covered. 

2.2  Evaluation 

Evaluators  in  the  static  format  evaluation  were  operational  crews 
from  the  318th  Tactical  Fighter  Squadron,  stationed  at  McChord 
Air  Force  Base  in  Washington.  A  total  of  twelve  pilots 
participated  in  the  evaluation.  By  rank,  they  were  one  Major, 
eight  Captains,  one  U.  S.  Navy  Lieutenant  and  two  First 
Lieutenants.  They  reported  305  to  3000  flight  hours  with  a  mean 
of  1776  hours.  Eleven  of  the  pilots  had  flown  F-15,  nine  T-38, 
seven  T-37,  four  F-106,  three  T-33,  three  AT-38B  and  one  each  F- 
4,  F-111A,  KC-135 ,  T-28 ,  T-39  and  T-43. 

The  briefing  began  with  an  explanation  of  the  purpose  of  the 
static  format  evaluation,  some  background  on  the  concept  of 
pictorial  formats,  and  an  explanation  of  the  questionnaire.  Each 
format  was  presented  first  in  its  color  version,  with  an  oral 
briefing  about  its  purpose  and  symbology.  Then  the  monochrome 
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version  of  the  toot  format  was  praaantad  with  an  explanation  of  I 

tha  monochrome  coding.  Finally,  aach  avaluator  completed  a  1 

questionnaire  for  a  givau  format  bafora  tha  next  format  waa  1 

l  -  ,<  i tad.  Both  versions  of  tha  format  wara  praaantad  on 

r  '  ad  projactora  along  with  tha  quaationnairaa,  and  questions 
i  tncouraged  during  tha  briafinga. 

Tha  firat  page  of  tha  questionnaire  requested  data  about  tha 
•valuator.  Bach  of  tha  ramaining  pages  aolicitad  raaponaaa  about 
ona  of  tha  formata  undar  avaluation,  and  waa  laballad  with  tha 
format  nama  at  tha  top.  Bxcapt  for  thia  lebal,  all  quaationnairo 
pagaa  wara  idantical. 

For  aach  format,  tha  firat  four  quaationa  raquirad  rating  tha 
format  on  fiva  point  acalaa.  Bvaluatora  ratad  aach  format  on  how 
uaaful  it  waa  for  ita  intandad  purpoaa  (from  "not  at  all  useful” 
to  "vary  useful" )}  how  aaay  it  waa  to  interpret  (from  "not  aaay" 
to  "vary  easy");  how  appropriate  tha  symbology  waa  (from  "not  at 
all  appropriate"  to  "vary  appropriate")}  and  how  tha  uaa  of 
color  affected  intarpratability  (from  "makes  interpretation  more 
difficult"  to  "makaa  interpretation  much  aaaiar").  Two 
additional  quaationa  aakad  what  nacaaaary,  but  currently  miaaing, 
information  ahould  be  added  to  tha  format,  and  what  unnacaaaary 
information  ahould  be  removed  from  tha  format.  Tha  final 
queationnaire  item  allowed  tha  avaluatora  to  make  general 
comment a  about  tha  format. 

2.3  Results 

Appendix  A  summarises  tha  pilot 'a  general  commanta  and  thoaa  • 

commant.a  which  apecifically  referred  to  tha  information  content 

of  tha  formata.  Average  ratinga  for  aach  format  are  shown  as 

prof ilea  in  Figure  2.3-1.  Raaponaaa  along  aach  of  tha  laballad, 

unnumbered  acalaa  wara  converted  to  ratinga  of  1  to  5,  where  1 

waa  tha  laaat  favorable  rating  (e.g.,  "not  at  all  uaaful"),  and 

wara  than  averaged.  With  a  few  exceptions,  tha  ratinga  i 

obtained  from  the  operational  crawa  wara  cloaaly  grouped  and 

favorable.  Results  for  each  of  tha  rating  questions  are 

discussed  in  tha  following  paragraphs. 

2.3.1  Usefulness 

Tha  alphanumeric  Electrical  and  Engine  Advisory  displays  received 

tha  highest  average  ratings  of  usefulness,  followed  closely  by 

the  ground  mode  HSP  and  PSF,  air  mode  HSF ,  Stores  Status,  and 

Passive  Sensor  Status  formats.  j 

( 

( 

I 

I 
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All  these  formats  rictivid  average  ratings  higher  than  4.0.  Only 
thraa  of  tha  sixteen  formats  received  average  ratings  of  lass 
than  3.0  ("boaewhat  useful")  on  this  question.  Thasa  formats 
wars  tha  ilaetrical  Status,  Bngina  Status,  and  Countermeasures 
Status.  Review  of  tha  comments  made  about  thasa  formats 
suggasts  soma  of  tha  reasons  for  tha  low  usefulness  rating. 
Several  avaluators  said  that  tha  Rlactrical  Systam  Status  format 
was  no  bat tar  than  tha  currantly  usad  talalight  panal. 

Similarly,  many  avaluators  preferred  standard  round  dial  or  gauga 
displays  to  tha  Bngina  Status  format.  Soma  avaluators  also 
emphasised  that  standard  angina  instrumants  and  a  talalight 
panal  for  systams  problams  display  information  continuously, 
which  is  an  advantage  ovar  tha  time-shared  pictorial  formats. 

2.3.2  Base  of  Interpretation 

Ratings  of  interpretabillty,  with  a  couple  of  exceptions,  ware 
similar  to  tha  ratings  for  usefulness.  Tan  of  tha  formats 
racalvad  average  ratings  greater  than  4.0}  thasa  included  tha 
advisory  formats,  most  of  tha  dynamic  situation  displays,  and  in 
contrast  to  tha  usafulnass  ratings,  tha  Countermeasures  Status 
and  Blactrical  Systam  Status  formats.  Only  tha  Bngina  Status 
forawt  received  an  average  rating  lower  than  3.0  ("somewhat  easy 
to  interpret").  Comments  suggest  that  tha  Engine  status  format 
was  rated  low  due  to  a  preference  ior  round  dial  instrumants  and 
tha  lack  of  numeric  readouts. 

2.3.3  Appropr lateness  of  Symbology 

As  expected,  ratings  of  symbology  appropriateness  closely 
paralleled  tha  ratings  for  ease  of  interpretation.  Tha  advisory 
formats  ware  among  nine  formats  that  had  average  ratings  higher 
than  4.0;  others  ware  tha  air  and  ground  mode  versions  of  tha  HSP 
and  PSP,  tha  Passive  Sensor,  Fuel,  and  Countermeasures  Status 
formats.  Again,  only  tha  Bngina  Status  format  received  an 
average  rating  lower  than  3.6  ("somewhat  appropriate").  Again, 
tha  low  rating  was  probably  due  to  a  preference  for  the 
conventional  instrumants. 

2.3.4  Use  of  Color  and  Interpratability 

Evaluators  generally  agreed  that  tha  use  of  color  made  the 
formats  easier  to  interpret.  All  sixteen  formats  received 
average  ratings  of  3.0  ("color  has  no  effect")  or  better,  and 
fifteen  of  the  sixteen  had  average  ratings  greater  than  4.0.  The 
average  ratings  on  this  question  were  highest  for  the  complex 
situation  displays;  the  air  and  ground  mode  PSP  and  KSF,  and 
Target  Formation  display,  and  for  the  detailed  Hydraulic  Status 
and  Passive  Sensor  Status  Displays.  Color  may  be  particularly 
useful  in  complex  or  detailed  displays  where  it  may  help  the 
viewer  to  sort  out  the  various  types  of  information  or  quickly 
identify  a  problem  area. 


2.3.5  Information  to  bo  Added  or  Dolotod 

Evaluate: 's  suggestions  about  information  to  bo  oddod  to  or 
dolotod  <.rom  tho  formats  ond  additional  comments  aro  summer ircd 
la  Appendix  A.  in  tho  table ,  each  suggestion  or  consent  la 
foilowd  by  tho  number  of  evaluators  who  node  that  reapente.  Tho 
evaluators  had  specific  au99oatlona  for  chan9oa  to  most  of  tho 
formats.  Many  of  thoao  suggestions  ware  implemented  in  tho 
format  rovlalona  dlacuaiod  in  tho  next  aoctlon.  Other 
au99oatlona  wore  not  implemented  bocauao  they  did  not  roproaont 
sufficient  conconaua  0x009  tho  ovaluatora)  because  they  wore 
antithetical  or  irrelevant  to  tho  objectives  of  tho  program)  or 
because  they  wore  imprecisely  defined. 

2.4  Application  of  Results 

The  static  format  evaluation  proved  to  be  of  significant  value  in 
the  development  and  evaluation  of  the  formats  used  in  the  MCFFD 
simulation.  The  first,  and  perhaps  not  so  obvious,  benefit  was 
the  early  development  and  production  of  high-quality,  computer¬ 
generated  versions  of  all  proposed  formats.  This  process  allowed 
the  rapid  generation  of  alternate  symbology  and  coding,  and  was  a 
powerful  and  accurate  tool  for  assessing  format  concepts  in  the 
design  and  revision  process  before,  during,  and  after  the  static 
format  evaluation. 

The  static  format  evaluation  itself  allowed  the  early  and 
effective  participation  of  operational  crews  in  the  format 
development  process,  and  resulted  in  the  confirmation  of  the 
validity  of  the  pictorial  display  concept,  especially  as  it 
applied  to  multi-crew  aircraft,  valuable  comments  by  the  crews 
led  to  the  incorporation  of  a  variety  of  changes  to  improve  the 
proposed  formats. 

On  the  RUD,  options  to  select  a  filled  or  unfilled  version  of  the 
pathway,  and  to  add  a  pitch  ladder  were  added,  along  with 
identification  of  weapon(s)  selected.  On  the  air  mode  PSF, 
optional  readouts  of  target  airspeed,  closing  rate,  and  range 
were  included  and  relative  altitude  symbology  was  deleted.  For 
the  hydraulic  status  format,  the  symbology  for  normal  systems  was 
changed  from  white  outline  to  green  fill)  this  change  made  the 
hydraulic  system  coding  more  similar  to  the  electrical  system 
coding.  Numeric  readouts  of  fuel  flow  in  pounds  per  hour  and 
percent  of  available  thrust  were  added  to  the  Engine  Status 
Format. 
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In  one  east,  coaaents  froa  the  operational  crews  in  the  static 
•valuation#  in  coabination  with  an  analysis  of  a  single-threat 
•eyond-Vi sue 1 -tenge  (SVA)  air-to-air  aission  setnario#  rssultsd 
in  the  development  of  a  now  forest.  Evaluators  requested 
readouts  of  target  altitude  and  airspssd  for  the  Tar gat 
Poraation  Display.  A  second#  tabular  version  of  the  Target 
Poraation  display  was  developed  to  display  iaore  detailed 
information  about  the  selected  targets.  These  two  versions  of 
the  Target  Poraation  Display  bee see  the  Poraation  and  Detail 
Close  Look  roraats. 
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3.0  TUT  EQUIPMENT  AND  PACZLITZIS 

The  Multi-Crew  Pictorial  Format  Display  simulation  was  conducted 
in  BMAC*s  Flight  Simulation  Laboratoriaa  in  Kant,  Washington. 
Thraa  of  tha  laboratoriaa  wars  uaad  in  this  simulation, 
intarconnactad  by  a  Pronat  10  MBs  communications  bus  ayatam. 
Pigura  3.0-1  shows  tha  major  alamants  of  this  simulation  and 
thalr  arrangamant.  Pigura  3.0-2  is  a  photograph  of  tha  two-saat 
simulator  cab.  Tha  following  paragraphs  dascrlba  tha  cockpit 
arrangamants  in  tha  two  saats  and  tha  configuration  of  hardware 
alamants  which  wars  employed  in  tha  simulation. 

3.1  Layout  of  tha  Two-Place  Fighter  Cab 

Tha  arrangamant  of  display  and  control  alamants  in  tha  front  seat 
is  shown  is  Pigura  3.1-1.  As  a  naming  convention  throughout  thin 
program,  tha  CATs  ware  called  "displays"  and  tha  pictures  shown 
on  those  displays  ware  called  "formats".  Thus,  tha  front  seat 
had  a  haad-up  display  (HUD)  as  wall  as  left,  center,  right,  and 
lower  multi-purpose  displays  (HPDs).  The  BUD  and  all  tha  MPDs 
ware  eight-inch  diagonal,  narrow  shadow  mask,  color  CRTs.  Tha 
fvD  combiner  was  removed  for  this  study.  Tha  MPDs  each  had  five 
uni aba 11 ad  push  button  switches  on  each  side  end  seven  labelled 
push  button  switches  below.  Tha  unlabelled  switches  ware  used 
in  conjunction  with  soma  of  the  formats.  Tha  labelled  switches 
under  tha  left  and  right  MPDs  were  used  to  select  tha  time-shared 
formats  and  those  under  the  canter  and  lower  MPDs  controlled 
options  on  tha  PSP  and  HSF,  respectively.  These  switch 
applications  are  detailed  in  tha  format  discussions  of  Section  4. 

Tha  panel  above  tha  left  MPD  contained  switches  for  air  mode  - 
ground  mode  selection  and  for  selection  of  several  HUD  options. 
Tha  panel  above  tha  right  MPD  contained  switches  for  stores  and 
countermeasures  selection.  Small  panels  to  the  left  and  right  of 
the  lower  MPD  had  switches  for  navigation  functions  and  cursor 
definition.  The  thrust  handle*  were  located  on  the  pilot's  left 
side  console  as  were  panels  for  the  fuel,  engines,  and  electrical 
systems. 

Both  seats  were  equipped  with  side-arm  controllers.  For  the 
pilot,  this  controller  was  used  as  the  primary  flight  controller. 
Bach  of  the  grips  for  these  controllers  had  a  two  way  trim 
switch,  a  trigger,  three  auxiliary  switches,  and  a  thumb  operated 
isometric  X/Y  controller  which  was  used  to  position  the  cursor  on 
the  horisontal  situation  format.  Except  for  the  HUD,  the  major 
front  seat  forward  panel  displays  and  controls  were  duplicated  in 
the  rear  seat,  as  shown  in  Figure  3.1-2. 
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FRONT  SEAT 
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Figure  3.0-1  Schematic  Diagram  of  Major  Simulation  Elements 


MAJOR  PANELS  AND  DISPLAYS 

A.  MASTER  MODE  PANEL 

B.  STORES  AND  COUNTERMEASURES 
SELECTION  PANEL 

C.  LEFT  MPD 

D.  CENTER  MPD 

E.  RIGHT  MPD 

F.  LEFT  AUXILIARY  SWITCHES  PANEL 

G.  LOWER  MPD 

H.  CURSOR  DEFINITION  PANEL 
i.  LOWER  LEFT  MPD 


Figure  3.1-2.  Reer  Seat  Panel  Arrangement 
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A  test  engineers'  station  was  located  above  and  behind  the 
cockpit.  From  the  station  the  test  engineers  had  a  direct  view 
of  crew  activities  during  a  simulation  run,  equipment  for  on-line 
data  monitoring  and  recording,  simulation  control,  and 
communications . 

3.2  Simulation  Host  Computers  and  Bus  Communications 

A  Gould  SEL  32/97  computer  and  a  Star  Technologies  ST-100  array 
processor  in  the  Visual  Flight  Simulator  Lab  performed  real  time 
modeling  of  airframes,  navigation  cells,  and  control  systems.  In 
addition,  they  supplied  graphics  subsystem  data  and  provided  on 
line  data  recording.  An  F-15  tactical  fighter  model,  a  real 
world  coordinate  navigation  cell,  and  a  flight  display  control 
program  provided  both  closed-loop  and  automatic  flight  mode'* 
Adversary  aircraft  and  all  airborne  missiles  were  also  mode  in 
the  host  computers.  Total  simulation  frame  time  was  less  than 
30  msec  after  the  addition  of  study  unique  graphic  control  logic, 
mission  profiles,  event  sequencing,  and  on  line  data  recording. 
The  specific  frame  time  for  any  given  display  format  depended  on 
its  complexity  (scene  or  symbology  content)  and  required  update 
rate  (30  He  for  flight  critical  and  1.0  to  30  Hs  for  non-critical 
information). 

The  digital  simulation  data  was  passed  through  a  10  KHs  serial 
Fronet  digital  data  bus  comprised  of  one  bus  controller  connected 
via  an  BED  interface  to  the  Gould  SEL  32/97  computer,  two  fiber 
optic  modem  units  which  transparently  interconnected  the  wire 
busses  in  each  facility,  and  two  bus  interface  controllers 
connected  via  memory  buffers  to  the  crewstation  I/O  system,  and 
the  Gould  SEL  32/67  graphics  generation  computer. 

3.3  Graphics  System 

Supported  by  a  dedicated  Gould  SEL  32/6780,  the  graphics 
generators  accepted  data  from  the  host  computers,  generated  the 
display  formats  for  the  HUD,  and  for  the  center  and  lower 
multipurpose  displays  and  updated  them  to  reflect  crew  input  and 
progress  of  the  mission. 

A  single  channel,  Kegatek  7250  color  graphics  generator  with  512 
by  512  pixel  resolution  was  used  for  generation  of  the  HUD.  Two 
of  the  three  channels  of  a  GTI  Poly  2000,  640  by  480  RGB  color 
raster  generator  provided  the  Close  Look  formats  and  the  three- 
dimensional  Perspective  Situation  Formats  on  the  center  hpds  in 
the  front  and  rear  seats.  The  Megatek  7250,  dual  channel  512  by 
512  RGB  color  raster  generator  drove  the  Horizontal  Situation 
Formats  in  the  front  and  rear  seat  lower  NPDs,  thus  minimizing 
data  handling  while  permitting  totally  independent  front/rear 
seat  display  manipulations.  The  Hegatek's  display  list  buffer 
was  doubled  to  permit  more  complex  horizontal  situation  formats. 


Three  Pioneer  laser  video  disk  units  contained  the  formats  for 
ths  front  and  rear  seat  multi-purpose  displays.  The  engine 
format*  which  time-shared  the  right  front  MPD,  was  programmed 
onto  a  dedicated  Heuricon  Graphics  board-level  generator. 
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4.0  PICTORIAL  FORMATS  AND  SIMULATED  SYSTEMS 

This  section  details  the  specific  aircraft  systems  and  the 
formats  which  serve  thea.  Soae  of  the  systeas  have  no  associated 
foraats  and  soae  are  represented  in  integrated  foraats.  In  a 
deliberate  inversion  of  systea  developaent  logic,  the  airplane 
sub-systems  were  developed  to  support  foraat  concepts,  rather 
than  the  other  way  around.  All  of  these  will  be  discussed  in 
turn. 


4.1  Flight  Control  Systea  and  AFCS 

Flight  controls  operated  in  the  conventional  manner. 

Hydraulically  actuated  surfaces  controlled  the  aircraft  in  three 
axes.  The  in-flight  speed  brakes  were  available  and  controlled 
by  a  switch  on  the  inboard  thrust  handle.  The  control  stick  and 
its  active  switches  are  shown  in  Figure  4.1-1.  Pitch  and  roll 
were  controlled  by  the  front  seat  stick.  Pitch  and  roll  trim 
were  controlled  by  the  trim  switch  on  the  front  seat  stick.  Trim 
did  not  relocate  stick  center.  Yaw  was  hydraulically  controlled 
with  the  rudder  pedals.  The  flight  control  stick  in  the  rear 
seat  served  only  as  a  site  for  the  switches  mounted  on  it. 

The  aircraft  had  a  unitary,  all  axis  autopilot  that  included 
autothrust.  It  was  selected  with  the  autopilot  button  on  the 
right  side  panel  and  deselected  with  either  that  button  or  the 
autopilot  disconnect  switch  on  the  stick. 

4.2  Primary  Display  System 

Display  formats  were  distributed  across  five  CRT  displays  in  the 
front  seat  and  four  CRT  displays  in  the  rear.  The  formats 
themselves  are  discussed  later  in  this  section.  The  HUD  was 
unique  to  the  front  seat  and  was  the  primary  flight  display. 

Each  seat  had  four  multipurpose  displays,  called  the  left, 
center,  right,  and  lower  MPDs.  The  Perspective  Situation  Format 
(PSF)  and  the  Close  Look  Formats  (CLFs)  time  shared  the  center 
MPD.  The  lower  MPD  was  the  site  for  the  Horizontal  Situation 
Format  ( HSF ) . 
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PITCH  AND  ROLL  TRIM 
AUTOPILOT  DISCONNECT 

WEAPON  RELEASE 


MIC  SWITCH 


Figure  4.1-1.  Flight  Control  Sticks 


A  number  of  system  related  formats  shared  the  left  and  right 
MPDs.  The  Stores,  Countermeasures,  and  Fuel  Status  formats  were 
available  on  the  left  MPD.  Formats  to  program  stores  and 
countermeasures,  and  advisory  formats  for  systems  health  problems 
were  also  available  on  the  left  MPD. 

The  Stores,  Countermeasures,  and  Fuel  Status  Formats  were 
available  on  the  right  MPD  as  well,  in  addition.  Electrical  and 
Hydraulic  Status  Formats  could  be  called  up  on  the  right  MPD  as 
could,  in  the  front  seat  only,  the  Engine  Status  Format. 

Lighted  push  button  switches  operated  with  many  of  the  formats. 
These  switches  had  three  states:  bright  for  "on",  green  for 
"option  available",  and  off  for  "option  not  available".  Function 
availability  was  determined  by  scenario  and  the  state  of  other 
mode  and  sub-mode  selections. 

4.2.1  Head-Up  Display  Format 

The  Head-Up  Display  (HUD)  was  the  primary  flight  instrument. 

Basic  flight  path  guidance  information  was  provided  by  the 
pathway  symbology  and  the  ownship  symbol.  The  pathway,  composed 
of  solid  white  segments,  served  as  the  directive  element  of  the 
display  providing  information  based  on  the  planned  route.  The 
wings  forming  the  entry  gate  of  the  pathway  functioned  as  the 
flight  director  and  the  ownship  symbol  served  as  the  velocity 
vector,  providing  heading  and  attitude  information.  Therefore, 
it  was  the  relationship  between  the  ownship  symbol  and  the  entry 
gate  of  the  pathway  which  provided  flight  guidance.  When  on 
course,  the  pathway  was  centered  about  the  ownship  symbol  and  the 
wings  of  the  ownship  were  aligned  with  the  wings  of  the  pathway. 
However,  once  ownship  deviated  from  the  planned  route  far  enough 
that  the  pathway  fell  outside  the  HUD  field  of  view,  the  pathway 
entry  gate  remained  at  the  edge  of  the  display.  To  provide  the 
pilot  with  steering  (pitch  and  bank)  commands  to  recapture  the 
planned  route,  a  transitional  flight  director  (a  white  inverted 
T)  which  moved  relative  to  a  reference  marker  (a  small  white 
square  centered  within  the  display)  was  added  to  the  display. 

The  transitional  flight  director  and  reference  marker  remained 
within  the  display  until  the  ownship  symbol  returned  to  the 
planned  route  with  the  correct  heading. 

In  addition  to  the  pathway  and  ownship  symbol,  Figure  4.2-1 
includes  features  displayed  on  the  HUD  when  they  were  within  the 
field  of  view  -  a  ground  plane,  a  zero  pitch  reference  line,  and 
terrain.  Generally,  the  terrain  and  pathway  moved  relative  to 
the  ownship  symbol.  Airspeed,  heading,  and  altitude  were 
presented  as  boxed  digital  readouts  at  the  left,  top,  and  right 
of  the  display,  respectively.  A  required  change  from  the 
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Ownship  Symbol 


Figure  4.2-1.  Basic  HUD  Symbology 


current  value  in  one  of  these  three  parameters  was  shown 
graphically  and  numerically  as  an  attached  solid  arrow  indicating 
direction  of  change  to  a  displayed  value.  The  roll  index  and 
indicator  were  located  along  the  lower  edge  of  the  display. 

HUD  options  available  to  the  pilot  included  the  pitch  ladder, 
pathway  fill,  and  a  vertical  velocity  indicator.  The  pitch 
ladder,  upon  selection,  was  added  to  the  pathway  and  the  ownship 
symbol.  Pathway  fill  replaced  the  solid  pathway  with  an  outline 
version,  rendering  the  pathway  transparent.  The  vertical 
velocity  indicator  as  added  to  the  display  included  a  digital 
readout.  For  the  HUD,  selection  of  a  Master  Mode  defined 
airspeed  in  knots  when  in  ground  mode  or  in  Mach  when  in  air 
mode. 

The  HUD  threat  alert  and  summary  information  was  shown  directly 
beneath  the  ownship  symbol.  When  an  airborne  threat,  surface-to- 
air  missile  site  (SAM),  or  anti-aircraft  artillary  (AAA)  site 
began  to  track  ownship,  an  aircraft,  missile,  or  gun  symbol 
appeared  in  an  alert  position  just  below  the  ownship  symbol  for 
six  seconds  and  then  shifted  into  the  summary  line.  The  summary 
line  showed  the  threat  type  and  number  of  threats  tracking  at  any 
point  in  time.  When  a  threat  launched  or  fired,  a  symbol 
returned  to  the  alert  position  and  flashed  for  the  duration  of 
missile  flight  or  the  firing  of  a  AAA.  Threat  site  asimuth  was 
indicated  by  the  clock  position  of  a  flashing  vector  radiating 
from  the  ownship  symbol.  For  an  inbound  missile,  a  time  to 
impact  and  missile  type  readout  was  Included  with  the  alert 
symbol . 

For  airborne  threats  and  targets,  missile  launch  envelope  (MLB) 
information  was  presented  when  ownship  or  a  target  was  tracked  or 
launched  upon.  The  attack  arrow  showed  the  capability  of 
ownship' s  selected  weapon  against  a  targeted  aircraft. 

Conversely,  the  defensive  arrow  displayed  the  assumed 
capability  of  the  adversary's  weapons  against  ownship.  Bach  of 
the  MLB  arrows  was  divided  into  four  sections  based  on  such 
factors  as  airspeed,  relative  geometry,  aspect  angle,  and 
maneuvering  capability,  in  addition  to  range.  The  top  section  of 
each  arrow  represented  a  son®  outside  the  weapon's  maximum 
range.  The  next  section,  a  zone  within  maximum  range  was 
followed  by  the  no-escape  zone.  The  no-escape  zone  was  defined 
as  within  the  effective  range  of  the  weapon,  such  that  a  target 
could  not  escape  thu  weapon  with  a  maximum  maneuver.  The  bottom 
section  of  each  arrow  was  a  zone  less  than  the  minimum  arming  and 
launching  range  of  the  selected  weapon.  A  particular  target  or 
threat  was  identified  with  a  numbered  caret  where  position  and 
movement  of  the  caret  along  the  MLE  arrow  was  indicative  of 
status. 
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figure  4.2-2  illustrates  ths  NLSs  and  ths  threat  alert  and 
summary  information  positioned  beneath  the  ownship  symbol.  The 
threat  summary  line  indicates  two  aircraft  are  currently  tracking 
ownship,  as  does  the  identified  carets  of  the  defensive  arrow. 
However,  ownship  remains  beyond  the  maximum  range  of  the 
threat's  weapons  as  shown  by  the  position  of  the  carets  along  the 
defensive  arrow.  The  threats  currently  tracked  by  ownship  have 
been  identified  as  Aircraft  1,  2,  5,  and  6,  are  within  maximum 
range  along  the  attack  arrow,  and  have  been  targeted  with  the 
long  AIM'S. 

Once  a  weapon  (air-to-air  or  air-to-ground)  was  targeted,  a 
weapon  type  and  number  readout  was  displayed  on  the  HUD.  A 
system  generated  cue  (an  X  within  the  ownship  symbol)  appeared  at 
the  optimal  weapon  release  point.  As  the  pilot  handed  the  weapon 
off,  using  the  trigger  on  the  flight  control  stick,  the  X  began 
to  flash,  then  vanishing  when  the  weapon  wgs  released. 

The  coding  or  color  coding  of  a  particular  symbol  as  described  in 
the  preceding  refers  to  both  the  color  and  monochrome  hud.  The 
two  versions  of  the  format  were  constructed  to  be  equivalent. 
Table  4.2-1  outlines  the  coding  of  the  basic  elements. 

Table  4.2-1 

Coding  of  HUD  Symbology 


HUD  Element 

Color  Coding 

Monochrome  Coding 

Pathway 

White 

Light  Grey 

Entry  Gate 

Cyan 

White 

Ownship 

Cyan 

White 

Ground  Plane 

Dark  Green 

Dark  Grey 

Terrain 

Light  Green 

Light  Grey 

Threat  Symbology 

Red 

White 

Attack  MLE  Arrow 

Within-Maximum-Range 

No-Escape 

White 

Green 

Medium  Grey 

White 

Defensive  MLE  Arrow 

Within-Maximum-Range 

No-Escape 

Amber 

Red 

Medium  Grey 

White 

Weapon  Release  Cue 

Cyan 

White 
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4.2.2  Perspective  Situation  Format 

The  Perspective  Situation  Format  (PSF)  displayed  the  planned 
flight  path,  terrain,  and  the  threat  environment  from  a  point 
6,000  feet  behind  and  1,000  feet  above  ownship.  The  basic 
elements  of  the  P8F  are  shown  in  Figure  4.2-3.  The  planned 
flight  path  consisted  of  a  series  of  white  triangles  which 
pointed  in  the  direction  of  flight.  The  ownship  symbol  was 
centered  within  the  display,  and  a  pyramid  symbol  on  the  ground 
plane  showed  the  point  directly  below  ownship.  The  ground  plane, 
overlaid  with  grid  lines  converging  on  the  horison  and  the  three- 
dimensional  surfaces  of  the  terrain  were  constructed  to  lend  a 
perception  of  depth  to  the  display.  Terrain  above  ownship' s 
altitude  was  differentially  coded  from  terrain  below  ownship' s 
altitude.  As  in  the  HUD,  airspeed,  heading,  and  altitude 
readouts  were  located  at  the  left,  top,  and  right  of  the  display. 

in  ground  mode,  active  surface-to-air  threats  were  depicted  as 
three-dimensional  lethality  volumes,  while  airborne  threat 
symbols  were  presented  in  an  abbreviated  form  (without  threat 
envelope  information).  AAA  sites  were  depicted  as  single  volumes 
of  uniform  lethality,  and  SAN  sites  consisted  of  outer  volumes  of 
moderate  missile  lethality  surrounding  inner  volumes  of  high 
lethality,  as  ownship  entered  these  volumes,  the  outer  surfaces 
of  the  envelope  folded  down  to  reveal  the  inner  volume  or  the 
footprint  representing  actual  ground  coverage  and  a  threat  site 
symbol.  Once  a  ground  threat  began  to  track  ownship,  a  lock-on 
circle  enclosed  the  ownship  symbol  and  a  tractor  beam  connected 
the  threat  site  to  ownship.  If  a  threat  launched  or  fired,  the 
tractor  beam  began  to  flash,  and  for  missiles  in  flight,  a  round 
missile  symbol  absorbed  the  tractor  beam  as  it  approached 
ownship. 

Distinguishing  SAN  sites  from  AAA  sites,  coding  of  threat 
lethality  envelopes,  and  terrain  above  and  below  current  ownship 
altitude  required  the  differential  use  of  color  and  shades  of 
grey.  Table  4.2-2  summarises  the  coding  of  monochrome  format 
elements  and  color  format  elements. 
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Table  4.2-2 

Coding  of  PSF  Symbology 


PSF  Element 

Color  Coding 

Monochrome  Coding 

Pathway 

White 

White 

Ownship  end 

Ground  Point  Pyramid 

Cyan 

White 

Ground  Plane 

Green  With 

Black  Grid 

Black  With 

Light  Grey  Grid 

Terrain 

Above  Aircraft  Alt<tude 
Below  Aircraft  Altitude 

Brown 

Green 

Dark  Grey 

White 

Sky 

Blue 

Black 

SAM  Site 

Inner  Volume 

Outer  Volume 

Red 

Amber 

White 

Medium  Grey 

AAA  Site 

Red 

White 

Tractor  Beam 

Red 

Light  Grey 

Lock  On  Circle 

Amber 

Medium  Grey 

The  PSF  air  Mode  symbology  of  Figure  4.2-4  provided  information 
concerning  gross  aspect  angle  and  aircraft  asimuth  and  elevation 
relative  to  ownship.  In  air  mode  surface-to-air  threats  were 
represented  only  by  the  site  symbols.  Aircraft  detected  by 
ownship  sensors,  were  shown  as  three-dimensional  airplanes  color 
coded  as  friendly,  unknown,  or  enemy.  When  they  were  in  search 
mode,  radar  coverage  sectors  directed  toward  ownship  were  added 
to  the  nose  of  the  aircraft.  For  an  aircraft  tracked  by  the 
Close  Look  Format,  readouts  of  radar  range,  closing  velocity,  and 
altitude  difference  were  displayed  in  the  lower  left  readout. 
Airborne  threat,  track,  and  launch  symbology  was  identical  to 
that  of  the  surface  threats. 

A  number  of  optional  features  allowed  each  crew  member  to  tailor 
his  own  PSF.  The  new  view  option  allowed  for  independent 
viewpoint  selection.  With  a  constant  slant  range,  the  viewpoint 
could  be  slewed  in  an  arc  from  a  nautical  mile  directly  behind 
ownship  (horizontal  view)  to  a  nautical  mile  directly  above 
(looking  down  in  a  vertical  view),  when  the  slave  option  was 
selected,  the  current  PSF  configuration  was  replaced  by  an  exact 
duplicate  from  the  other  crew  member's  display.  An  all-threats 
option  produced  threat  envelope  information  for  both  surface  and 
airborne  threats  regardless  of  the  Master  Mode  selected.  A 
preview  option  was  also  avail*? Is  in  conjunction  with  the  HSF ,  as 
described  in  paragraph  4.2.5. 
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Figure  4.2-4.  PSF  Air  Mode 


4.2.3  Horizontal  Situation  Format 

The  basic  Horizontal  Situation  Format  ( HSF )  was  a  plan  view 
display  consisting  of  the  planned  flight  route,  terrain  above 
current  ownship  altitude,  and  threat  information.  The  ownship 
aircraft  symbol  was  centered  within  the  display;  heading  was 
shown  as  a  digital  readout  at  the  top  of  the  display. 

The  forward  line  of  troops  (FLOT)  was  a  line  with  attached 
triangles  pointing  toward  enemy  territory.  Ground  targets 
appeared  as  white  triangles  and  waypoints  as  white  squares. 
Symbology  options  available  included  the  capability  to  change 
displayed  range  in  five  steps  from  20  to  320  NM;  the  capability 
to  add  or  delete  range  rings  and  fuel  rings;  and  the  capability 
to  move  ownship' s  position  to  the  bottom  of  the  display.  When 
displayed,  the  range  r'ngs  marked  a  distance  equal  to  one  quarter 
and  one  half  of  the  selected  display  range. 

The  fuel  range  rings  marked  normal  and  extended  fuel  range.  Time 
and  distance  to  the  next  waypoint,  next  target,  or  home  could  be 
optionally  selected.  As  in  the  PSF,  complete  symbology  for  all 
airborne  traffic  and  ground  threats  could  be  displayed  regardless 
of  the  master  mode  selected.  Figure  4.2-5  shows  the  symbology 
displayed  while  in  ground  mode:  the  FLOT,  the  ownship  symbol, 
range  rings,  the  planned  flight  route,  terrain  above  current 
ownship  altitude,  and  for  surface-to-air  threats,  complete  threat 
symbology.  In  ground  mode,  surface-to-air  threat  lethality 
envelopes  were  depicted  as  cross  sections  (at  current  ownship 
altitude)  of  the  same  three  dimensional  volumes  shown  on  the 
PSF.  AAA  sites  consisted  of  a  single  lethality  envelope  and  SAM 
sites  composed  of  a  core  of  high  lethality  surrounded  by  a  lower 
lethality  envelope.  As  shown  in  Figure  4.2-6,  four  threat  states 
were  depicted  for  surface-to-air  threats:  prebriefed,  active, 
track,  and  launch.  Prebriefed  threats  ~  inactive  threats  known 
only  through  reported  data  -  were  displayed  in  the  outline  form 
as  opposed  to  the  solid  form  of  active  threats.  As  in  the  PSF, 
once  a  threat  began  to  track  ownship,  a  tractor  beam  connected 
the  threat  site  to  the  ownship  symbol.  For  launching  or  firing 
threats,  the  tractor  beam  flashed,  and  for  SAMs,  missile  type 
(Infrared  or  Radar)  was  indicated  by  the  round  symbol  which 
absorbed  the  tractor  beam  as  it  approached  ownship.  while  in 
ground  mode,  reported  and  detected  aircraft  were  represented  by 
triangles  pointing  in  the  direction  of  flight.  Outline  triangles 
were  used  for  reported  aircraft  and  solid  triangles  for  those 
detected  by  ownship' s  sensors.  In  air  mode,  radar  coverage  as 
well  as  track  and  launch  symbology  was  added  to  the  aircraft 
symbols.  Complete  threat  coding  for  surface  threats  was  replaced 
with  abbreviated  symbols. 


Figure  4.2-6.  HSF  Ground  Mode  •  Threat  Symbology 


Ownship  was  assumed  to  ba  equipped  with  an  advancad  multimode 
sansor  systen,  operating  automatically  to  acquira  nacassary 
information  while  minimising  radiation.  Infrarad  Search  and 
Track  (XRST)  was  used  for  initial  detection  and  early  tracking) 
radar  was  used  only  whan  precision  was  required.  An  IRST  return 
was  displayed  as  a  dashed  line  which  extended  from  the  nose  of 
ownship  along  the  asimuth  of  the  return.  Once  the  detected 
aircraft's  range  was  adequately  refined,  the  IRST  return  line 
vanished;  only  the  detected  aircraft  symbol  remained.  When  in 
air  mode,  ownship  radar  coverage  was  always  displayed  in  some 
form.  If  ownship' s  radar  was  off,  the  potential  radar  coverage 
area  (120°)  was  shown  as  a  dashed  line)  if  ownship's  radar  was 
on,  the  radar  coverage  area  was  shown  as  a  solid  line.  As 
ownship  began  to  track  an  aircraft,  a  tractor  beam  was  added  to 
the  radar  coverage  area.  Placed  along  the  tractor  beam  were  the 
wi thin-maximum-range  and  no-escape  boundary  arcs  (as  in  the  HUD 
attack  MLE).  Weapon  status  information  was  shown  in  the  fora  of 
a  halo  enclosing  the  aircraft  when  targeted,  differentially  coded 
when  the  weapon  was  within  range. 

Upon  detection  by  ownship  XRST  or  radar,  an  unknown  or  an  enemy 
aircraft  was  displayed  as  a  solid  symbol  with  120°  of  potential 
radar  coverage.  For  an  airborne  threat  whose  radar  was 
searching,  potential  radar  coverage  was  reduced  to  a  ten  degree 
sector  of  actual  radar  coverage,  pivoting  from  the  aircraft's 
nose  toward  ownship.  When  an  aircraft  began  to  track,  radar 
symbology  was  replaced  with  the  tractor  beam  and  the  lock-on 
circle  enclosed  ownship.  Two  HLE  boundary  arcs  on  the  tractor 
beam  defined  the  within-maximum-range  and  the  no-escape  zones,  as 
in  the  HUD  defensive  RLE.  With  missile  launch,  the  tractor  beam 
flashed,  the  HLE  boundary  arcs  vanished,  and  a  round  missile 
symbol  absorbed  the  tractor  beam  as  it  approached  ownship. 

Figure  4.2-7  shows  the  HSF  symbology  displayed  in  air  mode. 
Ownship,  with  radar  on,  is  currently  tracking  a  hostile  aircraft 
(bearing  015°,  enclosed  by  a  pair  of  white  box  corners).  The 
hostile  aircraft  is  shown  as  detected  by  the  solid  symbol  with 
its  associated  potential  radar  coverage  area.  The  position  of 
the  target  aircraft  between  the  HLE  boundary  arcs  along  the 
tractor  beam  and  the  coding  of  the  halo  indicates  that  the  target 
is  within  the  maximum  range  of  the  weapon  assigned  by  ownship. 
(For  a  more  detailed  list  of  the  coding  conventions  used  in 
monochrome  formats  and  color  formats  refer  to  Table  4.2-3.) 
However,  a  second  hostile  aircraft  (bearing  340°,  enclosed  by  a 
set  of  four  cyan  box  corners)  is  tracking  ownship,  evidenced  by 
the  tractor  beam  and  lock  on  circle.  The  position  of  the 
defensive  HLE  boundary  arcs  indicates  that  ownship  remains 
outside  the  hostile  weapons  maximum  range. 


Tsble  C.2-3 

Coding  of  HSF  Symbology 


PSF  Element 

Color  Coding 

Monochrome  Coding 

Ownship 

Cyan 

Dark  Grey 

Manned  Flight  (tout* 

White 

White 

Ground  Mane 

Dark  Green 

Black 

Terrain 

Brown 

Light  Grey 

SAM  Sites 

Inner  Volume 

Outer  Volume 

Red 

Amber 

White 

Medium  Grey 

AAA  Sites 

Red 

White 

IRST  and  Radar  Symbology 

Cyan 

Dark  Grey 

Airborne  Traffic 

Friendly 

Unknown 

Hostile 

Green 

Amber 

Red 

Dark  Grey 

Medium  Grey 

White 

Lock  On  Greta 

Amber 

Medium  Grey 

Tractor  Beam 

Red 

White 

Attack  MLE  Boundary  Arcs 
Within-maximum-range 
No- escape  zone 

Green 

White 

Medium  Grey 

White 

Defensive  MLE  Boundary 

Arcs 

Within-maximum-range 
No-escape  zone 

Amber 

Red 

Medium  Grey 

White 

Weapon  Status  Halos 
Targeted 

Within  Range 

Amber 

Green 

Medium  Grey 

White 
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4.2.4  Clot*  Look  Po coats 

In  air  mode,  whan  tha  alccraw  raquirad  nora  datailad  information 
on  air  traffic  than  was  available  on  tha  BSP  and  PSP,  tha  Cloaa 
Look  Formats  (CLP)  wart  oalacted.  Tha  Pornation  CLP  was  an 
expansion  of  tha  araa  aalactad  fron  tha  HSP  and  tha  Datail  CLP 
was  a  tabular  arrangement  of  Xdantification  Friend,  roe,  or 
Neutral  (XFPN)  and  targeting  inforaation  on  aircraft  of  intaraat. 
To  display  an  aircraft  or  aircraft  foraation,  a  craw  aaabar 
engaged  the  HSP  cursor,  placed  tha  cursor  on  tha  appropriate 
syabol,  and  selected  tha  close  look  option.  Tha  Datail  CLP  was 
than  available  for  display  on  tha  canter  mpd,  replacing  the  psf 
air  node  upon  selection  of  tha  appropriate  switch. 

Once  selected,  tha  tabular  foraat  of  tha  Datail  CLP  displayed  tha 
XPFN  data  as  it  accuaulated.  Aircraft  ware  classified  with 
distinct  symbols:  enemy  aircraft  as  diamonds,  unknowns  as 
squares,  and  friendlies  as  circles.  Within  the  appropriate 
syabol,  multiple  aircraft  ware  represented  with  an  "M"  until 
more  precise  raid  count  information  was  available.  As  the  IFPN 
process  neared  completion  and  individual  aircraft  were 
identified,  specific  aircraft  type  was  noted  within  each  symbol. 
Direction  of  an  aircraft's  flight  was  indicated  by  a  vector 
attached  to  the  syabol  while  an  airspeed  or  relative  altitude 
readout  (crew  selectable)  was  displayed  above  each  syisbol.  Bach 
symbol  had  a  nominal  identification  (ID)  readout  beneath  it;  the 
same  ID  number  appearing  beside  a  switch  alongside  the  display. 

A  given  XD  switch  was  used  to  target  a  weapon  to  an  aircraft  or 
in  conjuction  with  the  track  function  of  the  Formation  CLP.  The 
ownship  heading  readout  was  located  at  the  center  top  of  the 
display.  A  switch  was  used  to  select  between  the  Detail  CLP  and 
the  Formation  CLP. 

Psing  the  same  nominally  identified  aircraft  and  the  basic 
symbology,  the  Formation  CLP  reflected  the  true  geometric 
relationships  among  aircraft.  While  lacking  some  specific 
information  concerning  individual  aircraft,  the  Formation  CLP 
reflected  the  behavior  of  the  formation  in  flight.  The  flight 
vectors  remained  attached  to  the  symbol,  while  specific  type  data 
within  a  symbol  and  the  peripheral  readouts  were  deleted.  An 
ownship  bearing  vector  was  attached  to  the  edge  of  the  display 
and  the  ID  numbers  for  each  aircraft  appeared  within  the  symbol. 
With  the  initial  selection  of  the  Formation  CLP,  the  display 
tracked  (or  centered  about)  a  single  aircraft  of  the  formation; 
other  aircraft  moving  relative  to  the  tracked  aircraft.  Th* 
option  to  select  a  different  aircraft  for  the  display  to  center 
about  was  accomplished  by  engaging  the  track  switch,  then 
pressing  an  ID  switch  corresponding  to  the  aircraft  ID  number. 
Another  option  allowed  the  size  of  the  close  look  window  to 
vary. 
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Figure  4.2-8  shows  ths  information  available  from  the  Formation 
or  Detail  CLFs  as  the  XFFN  process  nears  completion.  The 
formation  of  aircraft  selected  for  display  on  the  Detail  CLF 
consists  of  three  confirmed  hostile  F-39  fighters  with  an 
airspeed  of  Mach  1.8.  The  same  nominally  identified  aircraft 
appear  in  the  Formation  CLF.  The  positions  of  aircraft  ”2"  and 
*3"  are  displaced  relative  to  aircraft  Nl",  the  tracked  aircraft 
of  the  display. 

As  the  BVR  engagement  developed,  additional  coding  was  added  to 
the  symbols  of  both  the  Detail  and  Formation  CLF.  The  additional 
coding  (summarised  in  Table  4.2-4)  reflected  system  generated 
target  assignments  and  subsequently,  the  status  of  targeted 
weapons.  System  generated  target  assignments  for  both  wingman 
and  owr.thip  were  indicated  by  the  coded  rings  added  to  an 
aircraft  symbol.  With  selection  of  the  appropriate  ID  switch, 
effectively  targetting  the  selected  air-to-air  weapon,  a  solid 
halo  was  inserted  between  the  target  assignment  ring  and  the 
aircraft  symbol.  A  readout  indicating  the  type  and  number  of 
weapons  targetted  was  added  to  the  lower  edge  of  the  display.  A 
target  within  range  of  the  weapon  was  shown  by  the  differential 
coding  of  the  halo,  with  weapon  release,  the  target  assignment 
ring  vanished  while  the  weapon  status  halos  were  reduced  to  a 
thin  outline.  The  Formation  Close  Look  Formats  of  Figure  4.2-9 
illustrate  the  coding  of  the  symbology  as  target  assignments  are 
executed  and  weapons  released. 

Table  4.2-4 

Close  Look  Formats  -  Coding  of  Weapon  Information 


CLF  Symbology 

Color  Coding 

Monochrome  Coding 

Target  Assignment  Ring 
Owrnhip 

Wingman 

Cyan 

White 

White 

Dark  Grey 

Weapon  Targeted  Halo 

Amber 

Medium  Grey 

Weapon  Within  Range  Halo 

Green 

White 

Weapon  Release  Outline 

Green 

White 

4.2.5  Cursor  Functions 

A  number  of  cursor  options  were  independently  available  to  either 
crew  member.  The  general  procedure  involved  selection  of  the 
cursor  switch  from  beneath  the  HSF,  placement  of  the  general 
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Figure  4.2*8.  Close  Look  Formats 
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symbol,  followed  by  the  selection  of  an  option  from  the  cursor 
definition  panel.  It  was  froa  the  cursor  definition  panel  one 
accessed  the  Close  Look  Forests,  executed  the  route  segment  or 
cliab  profile  preview  function,  and  designated  targets  for  the 
air-to-ground  weapons.  The  options  available  at  any  given  point 
in  a  aission  were  a  function  of  the  current  state  of  the  HSF. 
Selection  of  an  option  defined  the  cursor  function  and  replaced 
the  general  cursor  syabol  with  the  functional  syabology  (refer  to 
Figure  4.2-10). 

While  in  ground  aode,  weapons  available  included  anti radiation 
Missiles,  boabs,  and  alnes.  The  anti radiation  missile  was  used 
as  a  defensive  weapon  to  counter  SAN  and  AAA  sites.  Once  the 
general  cursor  was  placed  over  the  ground  threat  (as  displayed  on 
the  HSF)  and  the  target  designated,  the  anti radiation  missile 
assignment  was  accomplished  and  an  unnumbered  target  symbol  was 
inserted  within  the  threat  site,  in  range  of  the  weapon,  the 
syabol  wa»  differentially  coded  and  the  weapon  delivery  cue  was 
added  to  the  ownship  syabol  of  the  HUD.  Bomb  and  mine  targets 
were  designated  by  using  the  cursor  to  mark  one  of  the 
preselected  target  locations. 

The  route  segaent  preview  function  allowed  either  crew  member  to 
preview  any  portion  of  the  planned  flight  path  in  order  to  assess 
the  threat  beddown  during  low  level  flight.  Once  the  preview 
option  was  engaged,  the  functional  symbology  within  the  HSF  and 
the  PSF  flew  the  planned  flight  path  at  faster-than-real  time 
until  the  preview  switch  was  selected  a  second  time  to  stop  the 
process.  The  displays  then  reverted  to  normal  real  time 
presentation.  In  a  similiar  manner,  the  climb  profile  preview 
function  allowed  either  crew  member  to  view  the  ground  threat 
environment  as  ownship  ascended  to  high  altitude  from  one  of 
several  points.  The  climb  profile  preview  function  was  available 
only  during  a  specified  portion  of  a  BVR  mission  segment  when  a 
set  of  system  generated  start  of  climb  points  were  displayed  on 
the  HSF.  The  general  cursor  was  then  placed  on  a  start  of  climb 
point,  the  preview  function  engaged,  and  the  functional  symbology 
flew  the  climb  profile  at  faster-than-real-time.  Upon  reaching 
the  top  of  climb,  the  symbology  ceased  to  move,  and  the  cursor 
could  be  selected  to  begin  the  procedure  again  for  preview  of  a 
second  or  third  profile.  Alternatively,  selection  of  the 
waypoint  (WAY  FT)  from  the  cursor  definition  panel  was  used  to 
insert  a  climb  profile  into  the  flight  plan. 

The  close  look  selection  cursor  option  was  available  once  air 
mode  had  been  selected  and  the  HSF  indicated  that  an  aircraft  had 
been  detected  (solid  symbology).  Once  selected,  the  generalized 
cursor  symbol  was  replaced  by  a  set  of  four  box  corners 
representing  the  area  displayed  on  the  CLF.  A  pair  of  box 
corners  was  used  to  indicate  the  other  crew  member's  close  look 
selection.  The  Detail  CLF  was  then  available  for  display  on  the 
center  HPD,  replacing  the  PSF  air  mode  upon  selection  of  the 
appropriate  switch. 
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Figure  4.2-10.  Cursor  Functions  Symbology 


4.3  Stores  System 

The  stores  carried  by  the  aircraft  supported  its  dual  mission 
with  both  air-to-air  and  air-to-ground  weapons,  tihile  similar  to 
existing  weapons#  the  weapons  had  features  not  in  the  current 
inventory.  All  were  guided  and  "launch-and-leave"  in  the  sense 
that,  once  fired,  no  further  input  was  required,  nor  was  there  a 
requirement  to  continue  to  illuminate  or  track  the  target. 

Four  long  range  air  intercept  missiles  (AlMs),  with  active  homing 
guidance  systems,  were  carried  for  the  air-to-air  engagements. 
Two  short  range  AZHs  and  a  gun  were  also  carried  but  not  used  in 
the  mission  scenarios.  The  two  defensive  antiradiation  missiles 
used  their  own  broad  band  seekers  for  in  flight  guidance.  The 
crews  were  briefed  to  employ  the  antiradiation  missiles  against 
unavoidable  threats  in  active  or\\track  modes.  The  aircraft 
carried  two  powered  homing  glide'  bombs  to  be  delivered  from  low 
altitude.  Bomb  guidance,  control,  and  propulsion  features 
allowed  deployment  against  a  variety  of  targets  in  a  relatively 
large  area  around  the  launch  poiiit.  \ The  bombs  had  an  automatic 
guidance  mode  in  which  they  accented  and  attacked  targets  at 
coordinates  established  with  the  target  cursor  option.  Operation 
of  the  aircraft's  two  mine  canisters  was  identical  to  that  of  the 
bombs . 

Bombs  and  mines,  as  part  of  the  preflight  procedure,  required 
programming  for  method  of  delivery,  guidance,  and  for  mines, 
burst  height.  From  the  stores  programming  menu,  available  on  the 
left  NPD  in  either  seat,  the  bomb  programming  menu  and  the  mine 
programming  menu  were  accessed.  Selections  were  made  by  pressing 
the  switches  beside  the  available  options.  Once  a  satisfactory 
selection  had  been  made,  selection  of  any  other  format  entered 
the  requested  options  into  the  system. 

The  Stores  Status  Format  (Figure  4.3-1)  showed  an  aircraft  plan 
view  with  the  inventory,  status,  and  location  of  onboard  weapons. 
One  short  range  AIM  was  located  at  each  wing  tip,  followed  by  an 
antiradiation  missile,  a  mine  canister  (aft),  and  a  glide  bomb 
(forward).  Located  along  the  centerline  were  the  four  long  range 
AIMS  and  in  the  nose,  the  gun. 

The  number  and  type  of  weapon(s)  selected  and  if  appropriate, 
additional  release  information  and  the  master  arm  off  indicator 
were  included  within  the  format.  Examples  of  the  Stores  Status 
Format  and  the  color  coding  used  to  reflect  the  state  of  the 
master  arm  and  the  selection  and  targeting  of  a  weapon  is  shown 
in  Figure  4.3-2.  As  a  weapon  was  selected,  the  body  of  the 
weapon  was  color  coded  and  an  outline  halo  was  added  to  the  nose 
of  the  weapon.  The  weapon  body  color  coding  represented  the 
status  of  the  master  arm;  the  color  coding  of  the  halo  was 
indicative  of  weapon  status.  Once  the  weapon  was  targeted,  the 
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Figure  4.3-1.  Stores  Status  Format 


Figure  4.3-2.  Stores  Status  Format  -  Selection  and 

Targeting  of  a  Weapon 


halo  was  shown  in  tha  solid  fora.  When  all  rslaass  parameters 
bad  been  satisified,  s.g . ,  required  range  and  master  arm  on,  both 
the  halo  and  the  weapon  body  coding  reflected  the  launch 
condition.  Table  4.3-1  summarises  the  coding  used  in  the  color 
and  monochrome  versions  of  the  Stores  Status  Format. 


T*bl94.3‘1 
Stores  Status  Coding 


Statue 

Weapon  Body  Coding 

Halo  Coding 

On  Board 

Coke- 

Outline 

None 

Not  Selected 

Monochrome 

Outline 

None 

Mutter  Arm  Off 

Cfffor 

Amber 

Outline 

Selected 

Monochrome 

Grey 

Outline 

,  * » 1  / 

Master  Arm  On 

Color 

Green 

Outline 

Selected 

Monochrome 

White 

Outline 

MasterArm  Off 

Color 

Amber 

Amber 

Targeted 

*  / » ' 

Monochrome 

Grey 

Grey 

Master  Anu  On 

Color 

Green 

Amber 

Targeted 

Monochrome 

White, 

Grey 

Master  Ami  Or? 

Color 

Green 

Green 

Within  Range 

Monochrome 

White 

White 

■ _ i 

Zn  addition  to  the  status  and  programming  formats  available  to 
either  crew  member,  control  of  the  stores  system  was  managed  froa 
the  stereo  selection  panel  in  conjunction  with  the  Close  Look 
Formats  sad  the  HSF.  However,  only  the  pilot  had  access  to  the 
master  arm  switch  and  only  the  pilot's  trigger  released  stores. 

As  previously  discussed,  stores  programming  was  accomplished 
with  the  format  on  the  left  MPD,  later  specific  stores  were 
selected  for  delivery  from  the  panel  above  the  right  HPD.  Long 
range  air  intercept  missiles  were  targeted  with  the  Close  Look 
Formats  on  the  center  MPD,  while  ground  targets  were  designated 
on  the  HSF  using  the  target  cursor  option. 

4.4  Countermeasures  System 

The  countermeasures  systea  consisted  of  a  radio  frequency  jammer, 
chaff,  and  flares.  The  jammer  operated  automatically,  going 
active  in  response  to  its  analysis  of  threat  states.  Chaff  and 
flares  were  programmed  .nd  dispensed  by  a  system  similiar  to  the 
current  an/ale  40. 

In  the  simulation,  the  aircrew  was  required  to  execute  the 
countermeasures  programming  tasks  as  part  of  the  preflight 
procedure.  The  countermeasures  information  and  control  menu  was 
accessible  to  either  crew  member.  As  shown  in  Figure  4.4-1,  the 
jammer  control  page,  the  chaff  and  flare  control  page,  the  flare 
information  page,  and  the  chaff  information  page  were  accessed 
from  the  information  and  control  menu.  The  options  available 
from  each  page  were  presented  as  a  row  of  selections  with  each 
row  aligned  with  one  of  the  side  switches  of  the  display;  the 
current  state  of  each  option  was  shown  by  a  box  enclosing  one 
selection  in  each  row. 

In  the  Countermeasures  status  Format  (Figure  4.4-2),  the  basic 
system  was  pictorially  presented  as  a  single  internally  mounted 
jammer,  a  stack  of  chaff  bundles,  and  a  stack  of  flares 
superimposed  on  an  ownship  planview.  With  chaff  and  flares  at 
levels  above  twenty-five  percent,  the  expendables  were 
automatically  released  as  required.  In  the  simulation,  the 
automatic  maximum  release  quantity  occurred  with  the  expenditure 
of  two  flares  or  four  chaff  bundles.  At  expendable  levels  below 
twenty-five  percent,  release  occurred  manually  at  a  reduced  rate 
(one  flare  or  two  chaff  bundles). 

The  jammer  was  coded  to  represent  three  states:  in  the  off  state, 
the  pod  and  bolt  were  in  the  outline  form,  while  in  standby  the 
bolt  was  shown  in  the  solid  form,  and  when  on,  a  pair  of  small 
lightning  bolts  appeared  outside  the  pod.  Once  the  jammer  was 
programmed  in  standby,  it  radiated  automatically  as  threats  began 
to  track  ownship,  reverting  to  standby  as  appropriate. 
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Figure  4.4-2.  Countermeasures  Status  Format 


The  status  formats  reflected  the  release  of  expendable 
countermeasures  as  previously  programmed.  At  levels  above 
twenty-five  percent,  flares  were  automatically  released  in  a 
burst  of  two  and  bundles  oi  chaff  were  expended  in  a  salvo  of 
four.  The  burst  of  flares  and  the  salvo  of  chaff  bundles  each 
comprised  one  row  within  their  respective  stacks,  in  the  normal 
status  format  sequence,  a  row  of  chaff  bundles  and  a  row  of 
flares  was  designated  as  "selected"  by  the  inclusion  of  a  small 
dot  within  each  symbol.  When  the  countermeasures  system 
determined  that  release  was  appropriate,  either  the  "released" 
row  of  flares  or  the  "released"  row  of  chaff  bundles  was  coded  as 
a  row  of  solid  symbols.  Once  "expended",  that  particular  row  of 
countermeasures  was  absent  from  subsequent  formats  and  the  next 
row  was  again  designated  as  "selected". 

In  the  simulation,  when  expendables  remaining  on  board  dropped 
below  the  twenty-five  percent  level,  the  system  automatically 
downselected,  limiting  the  release  of  countermeasures  to  two 
chaff  bundles  (or  one  flare).  The  low  quantity  situation  was 
indicated  by  illumination  of  the  master  caution  lights,  followed 
by  flashing  of  the  count? treasures  status  switch.  Figure  4.4-3 
illustrates  the  release  sequence  coding  that  occurred  within  the 
Countermeasures  Status  Format  subsequent  to  the  low  quantity 
situation.  Thereafter,  each  time  threats  launched  upon  ownship 
the  "selected"  countermeasure  was  color  coded  to  indicate 
"permission  required".  The  WSO  was  required  to  select  the  chaff 
or  flare  switch  in  order  for  the  expendable  countermeasures  to 
be  "released". 

4.5  Propulsion  System 

The  Engine  Status  Format,  available  only  to  the  pilot  on  the 
right  HPD,  displayed  propulsion  information  for  setting  thrust 
and  monitoring  engine  health.  The  format  consisted  of  two  plan 
view  engine  sections,  each  with  percent  thrust  shown  both 
numerically  and  graphically.  In  addition,  for  each  engine,  oil 
pressure  and  quanity,  exhaust  gas  temperature  ( EGT ) ,  afterburner 
operation,  fuel  flow,  and  state  of  the  fuel  valves  were 
displayed.  The  basic  Engine  Status  Format  and  its  major  elements 
arc  shown  in  Figure  4.5-1. 

Within  each  engine  body,  throttle  position  and  actual  thrust 
formed  a  vertical  bar  alongside  which  required  thrust  was 
indicated.  Throttle  position  was  represented  by  the  inner  twenty 
percent  of  the  thrust  bar.  Actual  thrust  was  represented  by  the 
remainder  of  the  bar.  Thrust  was  scaled  as  an  integrated 
measure  which  considered  all  relevant  engine  and  environmental 
parameters.  Thrust,  shown  numerically  in  the  nose  of  the  engine 
body,  was  registered  from  0  to  100%.  Afterburner  and  100%  thrust 
levels  occurred  at  the  lewer  (amber)  and  upper  (red)  indicating 
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Figure  4.5-1.  Engine  Status  Format 


lines*  respectively.  As  thrust  reached  the  afterburner  or  100% 
indicating  lines*  the  thrust  bar  was  appropriately  color  coded. 
Inquired  thrust,  calculated  as  a  function  of  required  speed,  was 
represented  by  a  pair  of  triangular  pointers  which  noved 
vertically  along  the  thrust  bar.  in  operation,  as  a  new  speed 
was  required  by  the  flight  plan,  the  triangular  pointers 
indicated  the  requirenent  along  the  thrust  bar.  The  engine  in 
the  first  diagram  of  figure  4.5-2  is  at  thirty-five  percent 
thrust,  but  the  triangular  pointers  show  that  more  thrust  is 
required.  The  pilot  then  moved  the  throttle  such  that  the  inner 
bar  was  level  with  the  position  of  the  pointers  as  shown  in  the 
second  diagram,  as  engine  thrust  spooled  up  or  down,  the  actual 
thrust  bar  rose  or  fell  to  match  the  throttle  position  and  the 
required  thrust  pointers,  in  the  third  diagram,  the  engine  has 
spooled  up  and  actual  thrust  has  increased  to  match  throttle 
position  and  required  thrust. 

The  symbology  used  to  present  the  various  states  of  oil  pressure, 
oil  quantity,  and  SOT  is  shown  in  Figure  4.5-3.  Oil  pressure  for 
each  engine  was  shown  as  a  pressure  gauge,  the  arrow  within  the 
gauge  pointing  up  for  good  pressure.  At  caution  level  and 
warning  level  underpressures,  the  arrow  pointed  left  and  down, 
repectively.  Oil  quantity  was  shown  as  a  reservoir  with  two 
horisontal  lines  defining  caution  and  warning  low  quantity 
levels.  BGT  was  represented  with  a  thermometer  of  vertical 
bars,  the  lower  afterburner  level  line,  and  the  upper 
over temperature  line.  For  the  purpose  of  the  simulation,  BGT  had 
four  discrete  states  per  engine:  with  an  engine  off,  the 
thermometer  was  empty;  normal  operation  occurred  below  the 
afterburner  level  line;  with  afterburner  operation,  the  level  of 
fill  rose  above  the  line;  overtemperature  was  indicated  with  the 
filled  thermometer  and  radiating  lines.  A  flame  at  the  rear  of 
each  engine  showed  the  state  of  the  engines,  a  small  flame  sysibol 
represented  normal  operations.  When  the  engine  was  in 
afterburner,  a  large  flame  enclosed  the  existing  symbol.  Fuel 
flow  was  represented  by  the  open  arrows  entering  the  engine  body 
shapes.  Amount  of  fuel  flow  was  shown  pictorially  by  the  level 
of  fill  within  the  arrows  and  numerically  (thousands  of  pounds 
per  hour).  Within  the  fuel  valves,  the  open  state  was  indicated 
with  the  vertical  lines;  in  the  closed  state,  the  horisontal 
lines  were  differentially  color  coded.  Valve  position  was 
changed  with  the  fuel  cut-off  switches  located  on  the  aft  side  of 
the  throttle  quadrant. 

Halfunctions  of  an  engine  were  indicated  by  the  illumination  of 
the  master  caution  lights  followed  by  the  blinking  of  the  engine 
status  switch.  The  status  format  indicated  the  nature  of  the 
problem.  All  actions  required  to  reduce  the  impact  of  the 
condition  were  included  within  the  advisory  format.  With  an 
engine  flame  out,  the  small  flame  symbol  vanished,  actual  thrust 
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Figure  4.5-2.  Required  Thrust,  Throttle  Position,  and  Actual  Thrust 


dropped  to  zero,  the  appropriate  EGT  thermometer  showed  empty, 
and  a  solid  warning  border  was  added  to  the  format.  With  engine 
damage,  EGT  temperature  was  elevated,  and  a  pair  of  warning  level 
markers  along  the  thrust  bar,  a  highlighted  nozzle,  and  a  striped 
caution  border  were  added  to  the  format.  The  advisory  format 
associated  with  engine  damage  instructed  the  pilot  in  the 
required  actions  and  cautioned  against  exceeding  afterburner 
limits  indicated  by  the  markers.  With  an  engine  fire,  a  fire 
light  was  illuminated,  requiring  a  response  by  the  pilot  to 
extinguish  the  light.  Within  the  status  format  a  larger  fire 
flame  was  added  to  the  affected  engine,  the  EGT  symbol  indicated 
an  overtemperature  condition,  actual  thrust  gradually  fell  to 
idle,  and  a  solid  warning  border  was  added  to  the  format. 

4.6  Aircraft  Systems 

4.6.1  Fuel  System 

Fuel  status  information  was  available  at  any  point  during  the 
simulation  with  selection  of  the  fuel  status  switch  beneath  the 
left  or  right  MPC.  The  basic  fuel  system  was  pictorially 
presented  as  seven  tanks  (two  wing  tanks,  two  inboard  tanks,  and 
three  centerline  tanks)  and  the  associated  fuel  lines, 
superimposed  on  an  ownship  plan  view  (Figure  4.6-1).  The  solid 
portions  of  a  tank  were  representative  of  fuel  on  board;  empty 
portions  of  a  fuel  tank  are  in  the  outline  form.  Consumption  of 
fuel  occurred  as  the  solid  portion  of  a  tank  was  gradually 
reduced.  An  alphanumeric  readout  was  used  to  indicate  total 
amount  of  fuel  remaining.  Simultaneously,  the  two  wing  tanks 
were  the  first  to  be  depleted,  followed  by  the  two  inboard  tanks 
and  the  centerline  tanks.  Fuel  flow  to  the  engines  was 
maintained  through  a  series  of  valves,  transfer  pumps,  and  boost 
pumps.  The  valves  were  represented  by  a  small  circle,  bisected 
with  a  fuel  line.  When  a  tank  was  depleted  and  there  was  no 
longer  a  functional  requirement  for  a  valve,  the  valve  was 
rotated  90°  and  the  associated  fuel  line  was  represented  in  the 
outline  form.  The  boost  pumps,  the  two  large  triangles,  fed  the 
engines.  Two  of  the  transfer  pumps,  the  smaller  triangles,  were 
located  within  the  two  inboard  tanks;  the  remaining  two  within 
the  centerline  tanks  Similiarly,  when  there  was  no  longer  a 
functional  requirement  for  a  transfer  pump,  it  was  absent  from 
the  format. 

Located  in  the  forward  seat  only,  left  side  panel,  were  the  fuel 
controls,  employed  primarily  as  system  malfunctions  occurred. 
Failure  of  an  element  within  the  fuel  system  may  have  resulted 
from  aircraft  damage,  therefore  the  following  symbology  occurred 
in  a  number  of  combinations.  When  one  of  the  four  transfer  pumps 
failed  (due  to  damage  or  a  system  malfunction),  a  striped 
warning  border  appeared  within  the  status  format  and  the  pufip  . 
symbol  was  differentially  coded.  Corrective  measures  included 
closing  the  associated  pump  valve  or  activation  of  the  cross  teed 
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Figure  4.6-1.  Fuel  Status  Format 


fuel  system.  In  the  event  of  a  boost  pump  failure,  the  failed 
pump  was  coded  and  the  automatic  emergency  boost  pump  (which 
until  required  did  not  appear  in  the  format)  was  activated  by  the 
system.  The  status  format  then  included  symbology  representative 
of  the  emergency  boost  pump  and  the  alternate  fuel  flow  lines.  A 
ruptured  tank  was  indicated  with  the  symbology  superimposed  over 
the  affected  tank.  Depending  on  a  tank's  location,  damage  often 
required  the  closure  of  the  fuel  tanks  associated  valves, 
effectively  isolating  the  damage.  The  numeric  total  indicator 
within  the  status  format  was  highlighted  with  color  as  the  fuel 
dropped  to  a  predetermined  critical  level  (bingo  fuel), 
indicating  to  the  crew  the  requirement  to  egress  to  a  refueling 
point. 


4.6.2  Hydraulic  System 

The  Hydraulic  Status  Format  (Figure  4.6-2)  showed  major  elements 
of  the  system.  Supported  by  four  subsystems,  the  aircraft 
provided  redundancy  for  \the  flight  critical  elements,  i.e.,  the 
canards,  leading  edge  flaps,  rudders,  elevons,  and  thrust  nozzle 
doors.  Non-flight  critical  elements,  i.e.,  the  canopy  release, 
gun  drive,  nose  wheel,  nose  wheel  steering,  main  landing  gear, 
and  aerial  refueling  probe  were  not  shown  as  redundant.  A 
specific  hydraulic  sybsystem  (1A,  IB,  2A,  or  2B),  functioning 
out  of  the  normal  range  of  operations  was  noted  with  an 
indicator,  while  the  individual  elements  were  appropriately 
coded.  A  failure  of  a  non-redundant ,  non-flight  critical 
element  was  coded  as  a  solid  cautionary  symbol.  A  redundant 
element  reduced  to  single  thread  operation  continued  to  function 
normally.  However,  to  distinguish  the  degraded  condition  of  a 
redundant  system  element,  a  cautionary  stripe  was  added  to  the 
symbol . 


4.6.3  Electrical  System 

The  Electrical  Status  Format  was  presented  as  a  high  level 
schematic  of  the  primary  elements:  the  buses,  generators, 
transformer  rectifiers,  batteries,  and  major  loads.  Each  element 
was  abbreviated  within  the  format  and  distinguished  by  shape. 
Electrical  system  health  problems  were  indicated  by  illumination 
of  the  master  caution  lights,  followed  by  flashing  of  the 
electrical  status  switch.  Within  the  status  format,  a  failed 
element  was  coded  at  the  caution  level  and  the  appropriate  relay 
switch  was  displaced,  if  appropriate. 

4.6.4  Passive  Sensor  System 

The  aircraft  was  equipped  with  a  passive  sensor  system  which 
provided  information  about  the  battle  environment  around  the 
aircraft.  The  Passive  Sensor  Status  Format  pictorially 
represented  the  system  as  a  wire  sphere  composed  of  six  sections: 
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Figure  4.6-2.  Hydraulic  Status  Format 


tpp  and  bottom,  each  possessing  a  forward  section,  a  near  side 
section,  and  a  far  side  section.  If  one  or  more  of  the  sensors 
were  degraded  or  failed,  the  area  of  coverage  lost  was  coded  at 
the  caution  level  and  connected  to  the  aircraft.  The  caution 
coded  areas  represented  zone?  from  which  the  aircraft  was  not 
receiving  complete  passive  sensor  information. 

4.7  Advisories,  Cautions,  and  Warnings 

The  advisories,  cautions,  and  warnings  in  this  simulation  were 
designed  to  exercise  the  system  status  formats  with  their 
attention  and  directive  features.  Thus,  the  procedures  required 
more  crew  activity  than  would  probably  be  the  case  in  a  new 
generation  aircraft.  While  the  formats  and  display  controls  from 
the  front  seat  were  duplicated  in  the  rear,  the  airplane  system 
controls  were  located  in  the  front  seat  only.  For  this  reason, 
all  of  the  crew  actions  associated  with  system  hoalth  problems 
were  executed  by  the  pilot.  When  a  system  malfunctioned  or  was 
damaged,  the  front  and  rear  seat  master  caution  lights  were 
illuminated. 

There  were  four  levels  of  malfunction,  with  increasing  levels  of 
criticality.  Level  I  was  advisory,  bringing  to  the  crew's 
attention  condition?  which  had  minimal  impact.  As  shown  in  the 
first  pair  of  status  formats  in  Figure  4.7-1,  the  advisory 
conditions  were  highlighted  by  the  coded  indicators.  Levels  II 
and  III  w<*re  cautious  (also  shown  in  Figure  4.7-1)  indicating 
that  the  system  health  problem  hnd  the  potential  to  adversely 
affect  the  aircraft  or  mission.  As  seen  in  the  second  and  third 
pairs  of  status  formats  of  Figure  4.7-1,  the  formats  included  a 
stripod  border.  The  difference  between  these  two  levels  was  that 
the  required  response  was  automated  in  Level  II ,  but  crew 
intervention  was  required  in  Level  III  problems.  Level  IV  was 
the  warning  level  where  the  status  formats  included  a  solid 
border  as  in  the  last  pair  of  formats  of  Figure  4.7-1.  Immediate 
crew  action  was  required  for  these  flight  critical  events.  Table 
4.7-1  summarizes  the  system  health  events,  in  terms  of  level  of 
criticality,  the  coding  of  the  indicators,  and  the  malfunctions 
or  damage.  Each  time  the  master  caution  lights  were  illuminated, 
the  speed,  heading,  and  altitude  readouts  and  the  ownship  symbol 
on  the  HUD  were  cclor  coded  at  the  caution  or  warning  level. 

A  uniform  procedure  was  executed  in  response  to  master  caution 
lights  for  system  health  problems.  The  pilot  action  was  to  press 
the  master  caution  light,  extinguishing  it,  select  the 
ap  iate  status  format,  and  report  the  problem  on  the  radio 

li..  At  Levels  III  and  IV,  once  the  basic  -esponse  procedure 

was  executed,  the  pilot  was  cued  to  the  available  advisory 
format.  The  pilot  selected  the  appropriate  advisory  format, 
followed  its  checklist,  and  reported  the  outcome. 
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Advisory  Caution  Caution  Warning 


Figure  4.7-1.  Advisory,  Caution,  and 

Warning  System  Status  Formats 


Table  4.7-1 

Levels  of  System  Health  Problems 


L«v«l 

HUD  Symbols 

Border 

Advisory 

Examples 

1  Advisory 

Color 

Amber 

None 

No 

Countermeasures  Low 

Minimal  Impact 

Monochrome  Reverse  Video 

None 

Quantity 

Bingo  Fuel 

II  Caution 

Color 

Amber 

Striped  Amber 

No 

C/L  Fuel  Tank  Damage 

Potential  Impact 

Monochrome  Reverse  Video 

Striped  Grey 

Boast  Pump  Failures 

Automatic 

Transformer  Rectifier  Failures 

Reconfiguration 

Battery  Failures 

Hydraulic  Subsystem  Failures 
Passive  Sensor  Sector  Failures 

III  Caution 

Color 

Amber 

Striped  Amber 

Yes 

Engine  Damage 

Potential  Impact 

Monochrome  Reverse  Video 

Striped  Grey 

Fuel  Transfer  Pump  Failures 

Crew  Action 

Fuel  Wing  Tank  Damage 

Required 

Generator  Failures 

IV  Warning 

Color 

Red 

Solid  Red 

Yes 

Engine  Fire 

Flight  Critical 

Monochrome  Reverse  Video 

Solid  White 

Engine  Out 

Immediate  Crew 
Action  Required 
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5.0  DYNAMIC  COMPARISONS 

The  second  planned  format  iteration  ended  with  the  dynamic 
comparison  evaluation.  It  was  conducted  after  the  formats  were 
collated  and  the  simulation  ready,  but  before  data  was  collected 
in  the  final  phases.  APNAL/FIGR  personnel  and  three  Air  Force 
pilots  were  briefed  on  the  simulation  and  formats,  then  the 
pilots  flew  the  simulation  repeatedly,  selecting  from  among  th«d 
offered  format  alternatives.  The  alternatives,  issues  and 
selections  are  summarised  in  this  section. 

5.1  Bead  Up-Display  Dynamic  Comparisons 

For  a  detailed  discussion  of  the  HUD  format,  please  see  paragraph 
4.2.1.  The  first  issue  on  the  HUD  was  arrangement  of  the  missile 
launch  arrows  in  the  air  mode.  This  is  illustrated  in  Figure 
5.1-1.  The  attack  (ATT)  arrow  on  the  left  side  represented 
ownship's  missile  launch  envelope  against  targetted  aircraft. 

The  logic  for  the  ATT  arrow  seemed  to  be  well  understood  and 
accepted.  The  scale  represented  by  the  arrow  included  range, 
known  capabilities  of  ownship  and  ownship's  selected  missile, 
hypothesised  capabilities  of  the  targetted  aircraft  and  the 
dynamics  of  the  evolving  engagement.  There  were  four  zones  in 
the  arrow,  representing  from  top  (head)  to  bottom  (tail):  beyond 
range  for  the  selected  missile,  in  range,  no-escape  for  the 
hypothesized  capability  of  the  targetted  aircraft,  and  too  close 
to  fire  the  selected  missile.  The  carets  representing  targetted 
aircraft  usually  started  near  the  top  (head)  of  the  arrow  and 
moved  down. 

The  issue  came  about  when  that  logic  was  inverted  to  create  the 
complementary  defensive  (DEF)  arrow,  representing  threat  missile 
launch  envelopes  against  ownship.  In  the  standard  version,  the 
DBF  arrow  had  the  same  four  zones,  representing  threat  aircraft 
and  missiles  against  ownship.  From  top  (tail)  of  the  DEF  arrow, 
they  were:  too  far  for  the  hypothesized  threat  to  fire  its 
hypothesized  missile  against  ownship,  in  range,  no-escape  for 
ownship,  and  too  close.  The  carets  for  threat  aircraft  usually 
started  near  the  top  (tail)  of  the  arrow  and  moved  down.  This 
way,  both  outbound  and  inbound  missiles  moved  in  the  direction  of 
the  arrows  and,  as  the  engagement  closed,  symbols  representing 
targets  on  the  ATT  arrow  and  threats  on  the  D5P  arrow  moved 
closer  to  ownship  at  the  bottom  of  the  arrows. 

The  offered  variant  for  the  HUD  DEF  arrow  had  the  zones  from  the 
top  (tail)  to  bottom,  represent  too  close,  no-escape,  in  range, 
and  too  far.  This  way,  ownship  was  at  the  tail  of  both  arrows 
and,  as  the  engagement  closed,  both  targets  on  the  ATT  arrow  and 
threats  on  the  DEF  arrow  moved  from  the  head  toward  the  tail  of 
the  arrows. 


Direction  of 
Symbol  Movement 
for  Closing 


STANDARD  OFFERED 

FORMAT  VARIANT 

Figure  5.1-1.  Alternative  Missile  Launch  Envelope  Definitions 


Tht  standard  format  was  selected,  on  ths  logic  that  it  would  be 
■ora  intuitivs  and  mors  easily  understood  for  the  target  and 
threat  aircraft  to  move  down  the  arrows  as  the  engagement  closed. 
The  arrows  both  pointed  in  the  direction  of  missile  flight. 

The  second  issue  on  the  HUD  was  how  much  fill  should  be  used  in 
the  format.  On  one  hand,  filled  areas  on  a  format  may  make 
those  areas  more  easily  understood.  On  the  other  hand,  excessive 
use  of  fill  consumes  more  energy,  takes  more  time  to  draw  a 
picture  frame,  and  puts  more  light  in  the  cockpit.  In  the  HUD, 
fill  covers  the  outside  scene.  Excessive  use  of  color  may  reduce 
its  attensity  value  for  important  symbology,  in  the  standard  HUD 
format,  both  mountains  and  the  ground  plane  were  filled,  the 
mountains  in  green  and  the  ground  plane  in  half  density  green  or 
full  and  half  density  gray,  in  monochrome.  The  offered  variants 
allowed  either  the  mountains  or  the  ground  plane,  or  both,  to  be 
left  black  framed  in  green  or  gray  lines,  it  was  decided  to  fill 
the  ground  plane  with  half  density  green,  since  this  is  an  IMC 
simulation.  In  visual  conditions,  a  declutter  feature  could  be 
employed  to  remove  the  fill. 

5.2  Perspective  Situation  Format  Dynamic  Comparisons 

The  perspective  situation  format  is  detailed  in  paragraph  4.2.2. 
The  dynamic  comparison  issues  again  related  to  use  of  color.  In 
the  standard  form,  the  sky  was  blue,  the  mountain  tops  were 
brown,  the  mountain  bases  with  the  ground  plane  were  green.  The 
ground  grid  was  black.  Offered  variants  had  the  sky  black,  the 
mountain  tops  white,  the  mountain  bases  light  gray  and  the  ground 
plane  black  with  a  green  grid.  The  more  colorful  standard  form 
was  selected  because  the  pilots  felt  that  the  full  colors  would 
be  more  meaningful  to  aircrews. 

5.3  Horizontal  Situation  Format  Dynamic  Comparisons 

The  first  issue  for  dynamic  comparison  on  the  HSF  again  was 
related  to  use  of  color.  In  the  standard  form,  the  mountains 
above  current  altitude  were  colored  brown  against  a  green 
background.  The  variant  had  light  gray  mountains  against  a  black 
background.  The  standard  format  with  brown  mountains  and  dark 
green  background  was  selected.  Some  of  the  smaller  symbols  were 
made  thicker  to  increase  contrast  against  the  colored  background. 

It  was  suggested  that  background  color  be  changed  as  a  function 
of  display  master  mode,  green  for  ground  mode  and  black  for  air 
mode.  This  idea  was  not  implemented  in  the  simulator. 


The  second  HSF  issue  concerned  missile  launch  envelope  symbology 
in  air  mode.  This  is  illustrated  in  Figure  5.3-1.  The  missile  ! 

launch  envelope  (MLB)  for  ownship  and  adversary  aircraft  could  be 
shown  with  either  reduced  or  full  representation.  Reduced 
representation  would  reduce  clutter  when  there  are  several  enemy 
aircraft  in  the  engagement  but  full  representation  may  be  more 
easily  understood.  In  the  standard  format  both  outbound  MLEs 
from  ownship  to  target  aircraft  and  inbound  MLBs  from  threat  to 
ownship  were  composed  of  a  line  between  aircraft  with  two  arcs 
defining  the  maximum  range  and  no-escape  zones.  The  offered 
variant  had  MLEs  with  10°  sectors,  color  coded  to  correspond  to 
MLE  sectors  in  the  HUD.  The  tractor  beam  representation  was 
selected,  based  on  the  clutter  argument.  The  pilots  felt  that, 
while  the  sectors  were  easier  to  see  and  understand,  the 
probab4lity  of  several  of  these  sectors  overlaying  was  high  and 
information  could  be  lost.  The  arcs  at  maximum  range  and  no¬ 
escape  range  were  made  thicker  to  increase  their  prominence. 


75 


76 


Figure  5.3-1.  Horizontal  Situation  Format  Alternatives 


6.0  CONDUCT  OP  THB  MISSION  SEONSNT  AND  COMPOSITE  MISSION  STUDY 

The  intent  of  the  manned  simulation  was  to  apply  tha  knowledge 
and  skills  of  operational  aircrews  to  assess  the  pictorial 
formats.  In  this  sense,  the  aircrews  acted  as  Measuring 
instruaents.  This  section  describes  what  the  aircrews 
experienced  in  their  three  days  on  site.  Briefly,  the  program 
began  with  ground  school,  followed  by  hands-on  training.  The 
crews  then  flew  mission  segments  where  the  emphasis  was  on 
collection  of  performance  data  in  a  variety  of  exercises  or 
events.  Finally,  the  aircrews  flew  composite  missions  with 
fairly  compressed  timelines  and  relatively  high  workload.  The 
emphasis  in  the  composite  missions  was  on  the  aircrews* 
subjective  assessment  of  the  pictorial  formats. 

6.1  Test  Subjects 

AFWAL/FIGR  arranged  for  sixteen,  two-man,  aircrews  to  serve  as 
subjects  in  this  study.  All  crew  members  were  active  duty 
military  aviators  and  most  were  current  in  one  or  more  combat 
aircraft.  Each  crew  consisted  of  a  pilot  and  a  WSO.  As  Table 
6.1-1  shows,  the  range  of  experience  and  aircraft  flown  was 
broad.  Names  and  addresses  of  participating  crew  members  were 
available  to  Boeing  approximately  two  weeks  before  their 
scheduled  test  dates.  A  handbook  (Martin,  Way,  and  Hornsby, 
1986),  designed  to  introduce  the  program,  the  formats,  and  the 
agenda,  was  forwarded  to  all  crew  members  two  weeks  prior  to 
their  arrival.  All  professed  familiarity  with  the  contents  of 
the  handbook. 

6.2  Mission  Segments 

The  purpose  of  the  mission  segment  simulations  was  to  compare 
usability  and  aircrew  acceptability  of  color  and  monochrome 
versions  of  the  pictorial  formats  unaer  controlled  conditions 
without  the  confounding  of  incompatible  simultaneous  task 
demands.  To  this  end,  aircrews  flew  three  different  sets  of 
mission  segments:  low  level  penetration,  system  health,  and 
beyond  visual  range  ( BVR)  air-to-air  engagement.  Events  and 
exercises  peculiar  to  a  given  segment  were  separated  in  time  so 
one  activity  was  completed  before  beginning  the  next.  There  were 
both  monochrome  and  color  practice  and  test  missions  in  each  set. 
Appropriate  performance  measures  were  collected  and,  at  the  end 
of  each  set,  crews  were  debriefed  on  the  formats  used  in  that 
set. 


77 


Table  6.1-1 

Aircrew  Qualifications 


Pilot 


Crtw 

Branch 

Jat  Hours 

Aircraft 

Jet  Hours 

Aircraft 

1 

AF 

2850 

T-38.F-15 

1500 

T-43,  F-4  E/G 

2 

N 

685 

T-2e.T-2.TA-4, 

EA-6B 

900 

EA-6B.TA-4J, 
F-4,  T-^9,  T-2 

3 

N 

2600 

A-6.  A-4,  T-2, 

T-34 

2300 

A-4.  A-6,  T-2, 
T-34 

4 

AF 

3155 

T-37,  T-38,  F-4, 
F-16 

3000 

T-43,  T-37. 
C-141.C-5 

5 

N 

2030 

EA-6B,  trainers 

875 

T-2C.T-39, 
A-4M,  EA-6B 

6 

N 

1325 

T-28B,  T-2C, 
TA-4.A-6E, 

C-172,  M-20C 

2520 

A-6,  T-39, 

RA-te 

7 

N 

325 

A-6E.TA-4.T-2C 

200 

A-6E,  TA-4,  T-47, 
T-39.  T-2 

8 

N 

1340 

T-34,  T-2C,  TA-4, 
A-6 

800 

A-6.T-39.T-2. 
A-4.  T-34, 

TC-4C,  TA-7 

9 

N 

3600 

T-34,  T-28,  T-2, 
TA-4,  A-4M,  A-6, 
EA-6B 

170 

T-34C,  T-2,  T-39, 
TA-4,  t  A-^B 

10 

AF 

2400 

F-15A/C.F-5  E/F, 
AT-38,  T-37, 

T-38,  F-4  C/D/E 

1800 

F-4.  F-111 

11 

AF 

4200 

T-37,  T-38, 
F-100.F-4.T-33, 
A-4,  f-38 
aggressors. 
F-TF-15,  VF-XX 

1425 

F-4,  T-29 

12 

AF 

3700 

KC-135.  T-39, 

F- 100,  F-4. 

F-104,  F-16, 

F-111 

2400 

F-4,  trainers 

13 

AF 

2900 

F-4,  F-16 

2500 

T-38,  A-10,  F-16 

14 

N 

2800 

A-6E.T-2C, 

A-4 

1300 

A-6E,  TA-4J, 
T-2C,  T-39 

15 

N 

1355 

F-14,  A-4,  T-2, 

T-34 

1400 

F-14,  F-4,  F-18, 
F-16,  A-6.S-3, 
P-3 

16 

N 

2400 

F-14,  F-4,  A-4, 

T-2,  fc-18 

1650 

F-14A,  TA-4, 
TA-7,  A-6,  T-38, 
TF-18 

Before  each  mission,  the  aircrew  worked  through  a  preflight 
checklist.  This  activity  ensured  proper  initial  conditions  for 
the  mission  and  served  as  a  review  for  the  crew  of  controls  and 
information  sources.  Part  of  the  checklist  included  programming 
of  the  stores  and  ECM  options  for  the  briefed  mission. 

6.2.1  Low  Level  Penetration  Segments 

The  low  level  penetration  mission  segments  began  at  terrain 
following  altitude  on  the  friendly  side  of  the  Forward  Line  of 
Troops  (FLOT).  Information  in  the  ground  threat  file  was 
relatively  sparse  at  the  start.  A  data  link  update  occurred  and 
new  threat  -'re  then  displayed  along  the  flight  path.  The 
flight  cre’v  was  to  select  a  minimum  exposure  route  at  several 
branch  points  as  the  aircraft  flew  through  the  mountain  passes. 
Crews  were  briefed  to  stay  close  to  the  selected  flight  path  but 
to  move  laterally  as  necessary  to  minimize  exposure  to  ground 
threats.  The  WSO  was  required  to  manually  approve  the  release  of 
chaff  and  flares  for  each  threat  that  launched  or  fired  upon  the 
ai  ri  raft . 

6.2.2  Systems  Health  Segments 

The  systems  health  mission  segments  began  at  terrain  following 
altitude  on  the  enemy  side  of  the  FLOT,  enroute  to  the  target 
area.  The  flight  ended  after  weapon  delivery  and  before 
transition  to  high  altitude.  As  in  the  low  level  penetration 
segment,  a  limited  ground  threat  file  was  updated  and  new  threats 
were  displayed  along  the  flight  path.  As  expendable 
countermeasures  were  also  limited  in  this  segment,  the  WSO  had  to 
manually  approve  the  release  of  chaff  and  flares.  Enroute, 
vhreats  launched  and  the  aircraft  sustained  battle  damage 
,  ecipitating  various  system  health  problems.  Each  segment 
contained  examples  of  failures  from  several  systems  (e.g., 
engine,  hydraulic,  fuel,  electrical,  countermeasures )  which 
requirer  appropriate  responses. 

6.2.3  Beyond  visual  Range  Air-to-Air  Segments 

The  BVR  air-to-air  mission  segment  began  during  low  level  flight 
and  ended  with  either  identification  of  friendlies  or  weapon 
delivery  against  a  group  of  confirmed  enemy  aircraft.  At  some 
point  in  the  flight  the  crew  was  directed  to  intercept  a  flight 
of  aircraft  whose  number  and  identity  were  not  known.  The  crew 
previewed  three  climbout  profiles  to  determine  a  minimum  exposure 
route  for  exiting  the  threat  area.  Ownship  then  flew  from  the 
start  of  climb  noint  to  the  intercept  point  where  the  IFFN/raid 
count  was  presented  and  if  appropriate,  ownship  engaged  the 
enemy . 


6.3  Composite  Missions 

The  navigation  track  for  the  composite  missions  is  shown  in 
Figure  6.3-1.  It  consisted  of  a  high  altitude  approach,  descent 
to  terrain  following  altitude,  penetration  through  a  moderately 
defended  area  to  an  interdiction  target,  target  acquisition, 
weapon  delivery,  withdrawal,  climbout  and  return  home.  The 
aircrew  encountered  air  and  surface  threats,  and  delivered  air- 
to-ground  and  air-to-air  weapons.  The  missions  were  flown  at 
altitudes  ranging  from  200  to  35,000  feet  AGL  and  airspeeds  from 
480  knots  to  Mach  2.2.  The  four  composite  missions  all  had  the 
same  structure.  Threat  beddowns,  target  assignments,  system 
health  events,  and  the  final  air-to-air  engagement  varied  from 
mission  to  mission.  A  more  detailed  outline  of  the  composite 
missions  follows. 

The  first  leg  began  just  after  aerial  refueling  enroute  to  WPl  at 
FL  200,  airspeed  496  knots  on  a  heading  of  330  degrees.  The  crew 
had  been  briefed  for  a  primary  and  a  secondary  ground  target. 
After  overflying  WPl  ,  the  aircraft  descended  to  200  feet  AGL  and 
entered  terrain  following/terrain  avoidance  mode  to  take 
advantage  of  the  terrain  masking  opportunity  offered  by  the  high 
ground. 

Entering  leg  B,  the  crew  received  a  data  link  message,  updating 
the  location  and  activity  of  known  ground  threats.  On  some 
missions,  there  was  a  directed  change  to  the  secondary  ground 
target.  Speed  was  maintained  at  496  knots  and  as  the  FLOT  was 
crossed,  the  onboard  passive  sensors  searched  for  enemy  radar 
activity  or  other  hostile  threats. 

There  was  an  opportunity  to  select  an  alternate  route,  based  on 
the  updated  threat  situation.  The  aircrew  made  overt  selections 
and  the  threats  countered.  The  aircraft  automatically  employed 
countermeasures  in  self  defense.  Some  battle  damage  occurred 
from  near  misses.  This  leg  was  heavily  defended  by  SAM  and  AAA 
mobile  units,  some  of  which  were  located  as  indicated  in  the  pre¬ 
flight  briefing,  while  others  popped  up  or  were  revealed  by  a 
data  link  update. 

On  leg  C  in  some  missions,  a  second  data  link  message  was 
received,  again  updating  the  threat  situation.  The  aircrew  again 
had  alternative  flight  path  choices  and  the  aircraft  sustained 
minor  damage.  Enroute  to  Waypoint  4,  the  crew  was  instructed  to 
attack  a  ground  threat  site  with  an  antiradiation  missile.  There 
was  1  ttle  terrain  masking  on  the  run  to  the  ground  weapon 
launch  point,  requiring  the  aircrew  to  be  particularly  alert  to 
threats.  The  WSO  targeted  the  weapon  and  the  pilot  released  the 
air-to-ground  ordnance. 
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On  some  missions,  the  crew  was  informed  of  aircraft  closing  on 
their  position.  The  direction  was  to  climb,  identify,  and  engage 
the  new  threat.  As  information  on  the  aircraft  accumulated, 
they  were  identified  as  friendlies  or  as  adversaries.  If  the 
other  aircraft  were  determined  to  be  hostile,  the  mission  ended 
with  a  BVR  air-to-air  engagement. 

6.4  Schedule 

The  complexity  of  the  aircraft  being  simulated,  the  novelty  of 
the  new  formats  being  evaluated,  and  the  amount  of  data  to  be 
collected  together  yielded  three  full  days  on  site.  Table  6.4-1 
shows  the  aircrew  agenda.  The  order  of  conditions  changed  from 
crew  to  crew. 

The  morning  of  the  first  day  was  devoted  to  ground  school  and 
cockpit  familiarization.  The  first  afternoon  consisted  of 
practice  flights.  On  the  morning  of  the  second  day  there  was  a 
briefing,  practice,  and  test  trials  on  the  first  mission  segment 
type,  followed  by  debriefing  on  that  segment  type.  That  pattern 
was  repeated  the  second  afternoon  and  the  third  morning  with  the 
second  and  third  segment  types.  The  final  afternoon  consisted  of 
flying  the  four  composite  missions  and  a  post  flight  debriefing. 

Table  6.4-1 
Aircrew  Agenda 


First  Day 

Ground  school 

Cockpit  familiarization 

General  flight  and  procedures  training 

Second  Morning 

First  Mission  Segment 

Briefing  and  demonstration 

Monochrome  or  color  display  condition 
Practice  segments,  one  test  segment 
Repeat  with  other  display  condition 
Debriefing 

Second  Afternoon 

Second  mission  segment  type 

Same  pattern 

Third  Morning 

Third  mission  segment  type 

Same  pattern 

Third  Afternoon 

Four  composite  missions 

Two  each  in  monochrome  and  color 

Final  debriefing 
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The  primary  independent  variable  was  format  presentation  - 
monochrome  or  color.  In  order  to  reduce  bias  introduced  by 
differential  learning,  the  odd-numbered  crews  flew  practice  and 
test  segments  in  monochrome  first,  then  in  color.  The  even- 
numbered  crews  reversed  the  order.  The  order  of  segment  sets  was 
also  varied  systematically  across  crews. 

The  order  of  flying  the  four  composite  missions  was  also 
counterbalanced  across  crews.  The  odd-numbered  crews  flew  their 
first  two  composite  missions  in  monochrome  and  the  second  two  in 
color.  The  order  was  reversed  for  the  even-numbered  crews. 

6.5  Data  Collection 

Three  types  of  data  were  collected  during  the  manned  simulations 
to  assess  the  usability  and  acceptability  of  pictorial  formats  in 
color  and  monochrome.  These  data  also  helped  improve  both  the 
content  and  depiction  of  aircraft,  flight,  .and  mission 
information.  The  three  data  types  were  questionnaire  or  opinion 
data,  subjective  workload  assessment,  and  performance  data. 

6.5.1  Questionnaire  Data 

Among  the  three  sets  of  mission  segment  flights,  i.e.,  low  level 
penetration,  systems  health,  and  BVR  air-to-air  engagement,  all 
of  the  individual  formats  were  exercised.  After  each  set, 
aircrews  was  asked  about  the  particular  formats  that  supported 
specific  events  during  that  set.  In  the  final  debriefing,  after 
all  the  mission  segments  and  composite  missions  had  been  flown, 
aircrews  responded  to  a  questionaire  on  suites  of  formats.  The 
emphasis  here  was  on  presentation  and  coordination  of  information 
across  formats.  Finally,  each  aircrew  member  was  provided  with 
a  tape  recorder,  paper,  and  a  list  of  general  questions.  It  was 
found  in  the  earlier  studies  in  this  series,  that  this  technique 
worked  well  to  elicit  ideas  not  otherwise  available  -  a  directed 
free  association.  Experience  has  also  shown  that  aircrew 
opinion,  collected  in  this  manner  and  collated,  is  extremely 
valuable  in  the  assessment  and  improvement  of  display  formats. 

6.5.2  Subjective  Workload  Assessment  Technique 

One  important  goal  in  the  design  of  these  formats  was  to  reduce, 
or  at  least  contain,  aircrew  workload.  The  Air  Force  has  had 
some  success  over  the  last  several  years  measuring  aircrew 
workload  with  a  program  called  the  Subjective  Workload  Assessment 
Technique  (SWAT)  (Reid,  1985).  During  the  course  of  this  study, 
aircrews  were  asked  to  use  SWAT  to  quantify  their  mental  workload 
required  to  complete  the  tasks.  Mental  workload  refers  to  how 


hard  the  crew  works  to  accomplish  some  task,  group  of  tasks,  or 
an  entire  job.  The  workload  at  any  one  time  was  assumed  to 
consist  of  a  combination  of  three  dimensions  which  contribute  to 
the  subjective  feeling  of  workload.  SWAT  defines  these 
dimensions  as  (1)  time  load,  (2)  mental  effort  load,  and  (3) 
psychological  stress  load. 

For  purposes  of  subjective  workload  assessment,  each  of  these 
three  dimensions  is  divided  into  low,  medium,  and  high  levels. 

The  SWAT  theory  assumes  that  each  workload  rater  may  have  a 
different  way  of  combining  the  three  dimensions  into  subjective 
workload.  For  example,  time  load  may  be  most  important  for  one 
rater  and  psychological  stress  most  important  for  another. 

In  order  to  be  able  to  combine  workload  ratings  across  raters,  it 
was  necessary  to  calibrate  the  raters  themselves.  Cards  were 
prepared,  each  containing  one  of  the  twenty-seven  combinations  of 
definitions  of  the  three  levels  of  the  three  dimensions.  Decks 
of  these  cards  were  given  to  the  individual  aircrew  members  with 
instructions  to  sort  the  cards  from  low  to  high  workload. 

Crewmen  were  asked  to  work  independently  and  they  took  from 
twenty  to  thirty  minutes  to  complete  the  task.  The  resulting 
orders  were  recorded  and  combined  by  a  psychometric  technique 
known  as  conjoint  analysis. 

During  each  mission  segment  flight,  twelve  tones  were  inserted  at 
places  intended  to  represent  a  wide  range  of  workload  conditions. 
Aircrew  members  were  instructed  to  report  workload  each  time  they 
heard  the  signal  tone.  The  pilot  reported  first,  follower'  by 
the  WSO. 

6.5.3  Performance  Data 

A  number  of  measures  of  crew  performance  were  recorded  during  the 
mission  segment  simulations.  Some  of  the  performance  measures 
reflected  individual  performance  of  the  pilot  or  the  WSO  while 
others  reflected  the  coordinated  activity  of  both  crew  members. 
Collectively,  these  measures  provided  quantitative  data  on  the 
crew's  ability  to  use  the  pictorial  information  in  color  and  in 
monochrome  to  accomplish:  a)  flight  path  control,  b)  threat 
detection  and  avoidance,  and  c)  identification  and  resolution  of 
degraded  system  status.  It  was  assumed  that  deviations  from  the 
flight  path  channel,  problems  in  threat  detection  and  avoidance, 
and  difficulty  in  the  identification  and  resolution  of  degraded 
mode  conditions  would  be  greater  for  a  display  presentation  mode 
that  was  more  difficult  to  use. 


7.0  RESULTS 


In  their  questionnaire  responses,  pilots  and  WSOs  clearly 
preferred  the  color  formats.  They  indicated  general  approval  of 
the  pictorial  format  concept  and  provided  detailed  critiques  of 
specific  formats.  Subjective  workload  assessments  did  not  show  a 
significant  difference  between  the  color  and  monochrome  formats. 
However,  there  was  an  apparent  learning  effect  favoring  the  color 
versions.  The  MANOVA  performed  on  performance  data  did  not  show 
significant  color/monochrome  difference.  However,  there  were 
weaknesses  in  aircrew  performance  which  could  be  identified  with 
particular  formats. 

7.1  Questionnaires 

Appendix  B  contains  profiles  of  usability  for  each  of  the  formats 
plotted  from  mean  judgements  made  by  the  aircrews.  Appendix  C 
contains  the  questionaires  with  mean  judgements  of  usability  and 
other  attributes.  Appendix  D  contains  summaries  of  the  aircrew 
responses  to  the  open  ended  questions. 

In  this  section,  the  formats  are  discussed  in  terms  of  aircrew 
ratings  of  usability  and  information  interpretability .  The 
individual  formats  are  discussed  in  terms  of  aircrew  ratings  of 
usability  and  information  interpretability.  Usability  ratings, 
were  given  on  a  seven  point  scale  from  1  fr.  *  "wry  good”  to  7  for 
"very  poor."  All  formats  were  rated  for  tb  :  c;lor  and  monochrome 
cases.  Two  general  observations  are  in  order.  First,  in  almost 
all  cases,  color  format  versions  were  rated  better  than  the 
monochrome  versions.  Exceptions  will  be  noted  as  they  come  up. 
Second,  the  data  profiled  in  this  section  show  that  most  of  the 
color  ratings  fell  between  2  ("moderately  good")  and  3  ("slightly 
good").  Symbology  elements  rated  less  than  "slightly  good"  may 
be  candidates  for  revision.  Similarly,  answers  to  information 
interpretation  questions  were  given  on  a  seven  point  scale  from  1 
for  "very  easy"  to  7  for  "very  difficult."  Elements  or 
information  rated  low  will  be  mentioned  here  and  considered  for 
revision  in  Section  8. 

7.1.1  Head-Up  Display 

Usability  ratings  for  HUD  elements  in  the  ground  and  air  modes 
are  profiled  in  Appendix  B  (B-2  and  B-3).  The  threat  alert 
symbols  in  both  modes  and  MLS  arrows  in  air  mode  were  given  high 
ratings.  The  MLE  arrows  also  received  strong  support  in  the 
open  ended  questions.  On  the  other  hand,  a  number  of  HUD 
symbology  elements  were  rated  down  and  suggest  revision. 
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The  pathway,  as  implemented,  was  not  well  received.  There  was 
support  for  the  concept  of  a  pathway,  but  the  present 
implementation  requires  a  significant  amount  of  work.  The 
transitional  flight  director  was  rated  less  than  "neutral"  in 
both  ground  and  air  modes.  The  pilots  were  not  happy  with  the 
addition  of  a  different  symbol,  with  different  control  rules, 
when  they  were  off  the  pathway,  in  ground  mode,  the  mean  ratings 
for  the  pathway  were  slightly  better  in  monochrome  than  in  color. 
Several  pilots  pointed  out  that  there  was  more  contrast  in  the 
monochrome  version.  The  airspeed  and  altitude  readouts  were 
acceptable,  but  the  heading  readout  was  less  so.  The  roll  index 
and  the  optional  vertical  velocity  indicator  were  also  not  well 
received. 

When  pilots  were  asked  to  rate  ease  of  information 
interpretation,  the  pathway,  the  vertical  velocity  indicator,  the 
weapon  release  cue,  and  the  relationship  of  ownship  to  terrain 
were  judged  to  be  less  than  "slightly  easy"  in  both  color  and 
monochrome.  Comments  indicated  that  a  nonlinear  scaling  would  be 
better  for  the  vertical  velocity  indicator,  to  yield  more 
sensitivity  at  low  vertical  velocities.  It  was  also  pointed  out 
that  the  weapon  release  symbol  in  the  HUD  should  be  more 
noticable  and  should  probably  not  be  an  "X,"  which  is  usually 
considered  to  be  a  breakaway  command.  Even  in  color, 
interpretation  of  ownship' s  location  relative  to  terrain  was 
reported  to  be  difficult. 

7.1.2  Perspective  Situation  Format 

Almost  all  of  the  elements  of  the  PSF  (Appendix  B-4  and  B-5)  were 
rated  better  than  "slightly  good"  in  color,  with  the  depiction  of 
ground  threat  lethality  volume  particularly  well  received.  In 
both  air  and  ground  mode,  the  preview  symbol  was  less  well 
accepted.  Crews  amplified  this  opinion  in  their  comments, 
indicating  that  the  preview  option  was  not  necessary.  The  ground 
grid  and  the  ground  point  symbol  were  down  rated  in  air  mode. 
Essentially,  they  carried  no  information  needed  by  the  crews  in 
air  mode.  The  new  view  option  -  the  ability  to  adjust  viewpoint 
on  the  PSF  -  drew  mixed  comments.  Some  pilots  and  WSOs  used  the 
feature  to  set  the  viewpoint  viewpoint  for  their  individual 
preference  or  to  meet  their  needs  for  a  particular  mission 
segment.  Other  pilots  and  WSOs  did  not  use  it  or  did  not  see  the 
need  for  this  feature.  While  speed  and  altitude  readouts  were 
fairly  well  accepted,  in  air  mode  the  heading  readout  was  rated 
worse  than  "slighc  j  good." 

For  information  interpretation,  WSOs  rated  airborne  threat  mode 
and  type  information  less  than  "slightly  easy."  Pilots  rated 
airborne  threat  type  and  the  numeric  data  on  airborne  threats 
less  than  "slightly  easy." 


Comparing  ratings  and  opinions  on  the  PSF  in  this  study  with 
those  in  the  earlier  pictorial  format  display  evaluation  (Way,  et 
al,  1964),  this  is  an  improvement.  The  primary  differences  are 
that  threat  depiction  was  more  complete  in  this  study  and  the  PSF 
in  ground  mode  had  a  purpose  here  -  simultaneous  threat  and 
terrain  avoidance. 

7.1.3  Horisontal  Situation  Format 

Usability  ratings  of  the  HSF  in  ground  and  air  modes  are  profiled 
in  Appendix  B  (B-6  and  B-7).  With  the  exception  of  the  fuel 
range  rings,  all  the  symbology  elements  in  color  were  rated 
better  than  "slightly  good."  In  amplifying  comments,  the 
aircrews  questioned  the  utility  of  the  fuel  range  rings. 

The  pilots  rated  the  fuel  range  rings  and  ownship's  proximity  to 
mountains  worse  than  "slightly  easy"  to  interpret.  The  WSOs 
passed  those  but  downrated  enemy  KLE  boundary  arcs  on  the  HSF  in 
air  mode. 

Despite  these  specific  complaints,  most  of  the  pilots  and  WSOs 
reported  that  the  HSF  was  an  excellent  format  which  provided  a 
good  awareness  of  the  current  and  near  future  situation.  When 
asked  if  the  HSF  background  color  should  change  as  a  function  of 
master  mode,  e.g.,  black  for  air  mode  and  green  for  ground  mode, 
the  clear  majority  said  "No." 

7.1.4  Close  Look  Formats 

Crew  ratings  of  the  Detail  and  Formation  versions  of  the  Close 
Look  Format  are  profiled  in  Appendix  3  (B-8).  In  color,  all  but 
one  of  the  symbology  elements  were  rated  better  than  "slightly 
good."  The  one  feature  which  was  rated  down  was  the  range  change 
feature  of  the  Formation  CLF.  The  utility  of  this  feature,  as 
implemented,  was  questioned.  Crosschecks  between  the  CLFs  and 
HSF  and  between  the  CLF  and  Stores  Status  formats  were  rated 
"moderately  easy"  by  the  pilots  and  WSOs.  Ability  to  interpret 
information  on  the  CLFs  was  also  rated  "moderately  easy." 

Although  it  does  not  appear  in  the  formal  data,  a  number  of 
crewmen  commented  that  having  two  Close  Look  Formats  was  awkward, 
leading  to  occasional  confusion. 

7.1.5  Stores  and  Countermeasures  Formats 

The  Stores  and  Countermeasures  Status  and  Programming  Formats 
were  rated  better  than  "slightly  good"  in  both  color  and 
monochrome  as  indicated  in  Appendix  B  (B-9).  In  their  comments, 
crews  indicated  general  satisfaction  with  these  four  formats. 
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7.1.6  Othir  Status  Formats 


Usability  ratings  for  symbology  elements  in  the  Engine  Statue 
Format  are  profiled  in  Appendix  B  (B-10).  In  color,  all  the 
elements  except  the  fuel  flow  arrows  were  rated  "slightly  good" 
or  better.  In  their  comments,  pilots  indicated  that  numeric  fuel 
flow  was  sufficient  and  that  the  redundant  fill  level  in  the 
arrows  was  not  necessary.  They  indicated  that  simplification  of 
the  format  would  make  important  information  easier  to  find.  The 
color  versions  of  the  Fuel,  Electrical,  and  Hydraulic  Formats  and 
their  elements  were  rated  better  than  "slightly  good"  as  shown  t\\ 
the  usability  ratings  profile  of  Appendix  B  (B-ll).  Finally, 
usability  ratings  for  elements  in  the  Passive  Sensor  Status  and 
System  Advisory  Formate  are  shown  in  Appendix  B  (B-12).  The 
color  versions  of  these  were  rated  "slightly  good"  or  better. 

7 . 2  Workload 


The  Subjective  Workload  Assessment  Technique  (SWAT)  was  used  as 
the  measure  of  workload.  SWAT  measures  require  two  stages  of 
scale  development  and  event  scoring.  Workload  is  defined  as  an 
aggregate  of  three  factors:  (1)  time  load,  (2)  mental  effort 
load,  and  (3)  psychological  stress  load. 

Scale  development  began  by  having  each  of  the  thirty- two  subjects 
(sixteen  pilots  and  sixteen  WSOs)  sort  a  deck  of  cards  to  reflect 
their  individual  perception  of  how  the  three  factors  combine  to 
determine  workload.  Each  of  the  cards  contained  one  of  the 
twenty-seven  combinations  of  three  levels  (low,  moderate  and 
high)  of  the  three  factors.  This  process  typically  took  a 
subject  about  20  to  30  minutes.  The  card  order  for  each  subject 
was  recorded.  A  preliminary  scale  wa?  derived  using  a 
statistical  process  known  as  conjoint  analysis  (Nygren,  1982). 
Kendall's  Coefficient  of  Concordance  (W)  was  calculated 
indicating  a  moderate  level  of  agreement  among  the  32  subjects 
on  that  scale  (W  -  .72).  Following  a  procedural  suggestion  of 
Reid,  Eggemeier  &  Nygren  (1982),  the  individual  subjects  were 
prototyped  or  characterised  as  effort,  stress  or  time  responders, 
depending  on  their  card  sorts.  New  scales  were  derived  based  on 
those  prototypes.  Four  of  the  subjects  were  characterized  as 
effort  responders  (W  -  .87),  five  as  time  responders  (w  ■  .94) 

and  the  remaining  twenty-three  as  stress  responders  (w  -  .82).  A 
linear  transformation  was  applied  to  yield  a  scale  from  0  (no 
workload)  to  100  (maximum  workload). 

The  aircrews  flew  their  six  test  missions  -  low  level 
penetration,  system  health  and  beyond-visual-range  air-to-air  - 
each  with  monochrome  and  color  formats.  Twelve  auditory  tones 
were  placed  in  each  of  the  missions  -  six  during  busy  times  and 
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six  during  less  busy  tines.  The  crews  were  instructed  to  give  a 
SWAT  judge sent  when  they  heard  a  tone.  Their  judgement  data  was 
scaled,  using  the  process  just  described  and  then  subjected  to  an 
analysis  of  variance.  Table  7.2-1  summarized  that  analysis.  The 
independent  variables  were: 

Mission  -  low  level  penetration,  system  health  or  beyond- 
visual-range; 

Display  -  monochrome  or  color  formats; 

Crewman  ••  pilot  or  WSQ ; 

BNB  -  an  independent  division,  by  the  experimenters, 

of  the  twelve  SWAT  occasions  in  each  mission 
into  the  six  busiest  and  the  six  least  busy;  and 

Sequence  -  a  division  of  the  sixteen  aircrews  into  the 

eight  who  were  tested  on  color  before  monochrome 
and  the  eight  who  were  tested  in  the  other 
order. 

Table  7.2 - 1 .  SWA  T  Reports  -  Analysis  of  Variance  Summary  Table 


Source 

or 

TypeISS 

F  value 

Mission 

2 

25841  .88 

26.53* 

Display 

1 

540.92 

1.11 

Crewman 

1 

26896.25 

55.22* 

BNB 

1 

90967.97 

186.75* 

Sequence 

1 

100725.42 

206.78* 

Mission  by  display 

2 

2945.89 

3.02* 

Mission  by  crewman 

2 

16087.02 

16.51* 

Mission  by  BNB 

2 

1601.55 

1.64 

Mission  by  sequence 

2 

1299.45 

1.33 

Display  by  crewman 

1 

13.93 

0.03 

Display  by  BNB 

1 

70.90 

0.15 

Display  by  sequence 

1 

378.18 

0.78 

Crewman  by  BNB 

1 

2399.96 

4.93* 

Crewman  by  sequence 

1 

3770.79 

7.74* 

BNB  by  sequence 

1 

1  r3$0.47 

23.79* 

Mission  by  display  by  crewman 

2 

1510.12 

1.55 

Mission  by  display  by  BN8 

2 

182.99 

0.19 

Mission  by  display  by  sequence 

2 

1735.92 

1.78 

Display  by  crewman  by  BNB 

1 

1.78 

0.00 

Display  by  crewman  by  sequence 

1 

317.36 

0.65 

*  Significant,  p<. 05 
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Table  7.2-2  gives  the  mean  SWAT  scores  for  the  main  effects.  The 
scores  were  significantly  lower  for  systen  health  aissions  than 
the  other  two,  for  the  WSO’s  than  the  pilots*  for  the  "not  busy" 
occasions  than  the  "busy"  ones  and  for  crews  who  were  tested  on 
color  before  monochrome  than  those  who  were  tested  in  the  other 
order.  The  small  display  difference  was  not  significant. 
Significance  was  tested  by  the  Bonferroni  (Dunn)  t-test  within 
the  8A8  General  Linear  Models  Procedure,  a  criterion  of  0.05  was 
selected  for  the  significance  deteraination. 


TM*  7.2-2.  SWAT  Reports  -  Main  Effect  Means 


Uitn 

level* 

Moan  SWAT  Scorts 

Minim 

LLP.  BVR,  SH 

25.7U 

24.06 

17.89 

Display 

Color,  monochrome 

22.89 

21.88 

Crewman 

Pilot  WSO 

26.00 

18.68 

•NS 

Busy,  not  busy 

29.41 

15.74 

Sequence 

Monochrome  first  color  first 

29.57 

15.26 

The  significant  sequence  difference  is  particularly  interesting. 
Crews  who  were  trained  and  tested  on  color  foraats  before 
monochrome  formats  reported  overall  mean  workload  14.31  scale 
units  lower  than  crews  with  the  other  order.  This  seems  to 
Indicate  that,  although  the  color  -  monochrome  effect  was  not 
itself  significant,  it  participated  in  an  order  or  learning 
effect  on  all  three  segment  types  and  with  both  pilots  and  wso's. 

Even  after  an  intensive  training  program,  the  short  time 
available  caused  the  crews  to  be  still  learning  as  they  were 
being  tested.  This  supports  the  interpretation  of  order  effects 
as  learning.  Apparently,  the  differential  transfer  from  color 
formats  to  monochrome  yielded  lower  overall  perceived  workload 
than  the  transfer  from  monochrome  to  color. 

Figure  7.2-1  contains  plots  of  the  significant  two-way 
interactions.  With  a  significant  mission  main  effect,  but  no 
significant  display  effect,  the  significant  interaction  appears 
to  be  a  modest  inversion  of  workload  in  the  beyond-visual-range 
mission  and  functionally  unimportant.  Both  crewman  and  mission 
main  effects  were  significant.  The  significant  interaction 
between  them  indicates  that  the  pilots  and  WSO's  perceived  their 
mental  workload  to  be  almost  the  same  in  the  BVR  missions,  but 
pilots  reported  higher  workload  than  WSO's  in  the  other  missions. 
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Figure  7.2-1.  SWAT  Reports  -  Significant  Two-way  Interactions 
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The  three  interactions  in  the  lower  row  of  figure  7.2-1  have  the 
same  pattern  and  appear  to  support  the  sane  interpretation,  in 
each  case,  the  parameter  with  the  lower  score  has  a  shallower 
slope.  This  appears  to  be  a  floor  or  scale-end  effect  where 
scores  near  the  extremes  of  a  closed-end  scale  tend  back  toward 
the  center. 

7.3  Performance 

The  performance  measures  collected  in  the  low  level  penetration 
(LLP),  system  health  (SH),  and  beyond  visual  range  air-to-air 
(BVR)  test  segments  can  be  broadly  catagorised  into  three  groups: 
flight  path  control  data,  latency  of  the  required  responses,  and 
the  frequency  of  response  error.  A  subset  of  the  three  types  of 
performance  measures  were  appropriate  in  each  of  the  three 
mission  segments  as  indicated  in  Table  7.3-1. 

Flight  path  control  performance  measures  generated  at  the 
conclusion  of  each  trial  were  defined  as  follows. 

Root  Mean  Square  (RMS)  Error,  Vertical:  A  cumulative 
measurement  of  the  vertical  deviation  from  the  displayed 
flight  path. 

RNS  Error,  Lateral:  A  cumulative  measurement  of  the 
horisontal  deviation  from  the  displayed  flight  path. 

Percent  Time  in  Pathway,  Vertical:  Percentage  of  time  the 
flight  path  was  flown  within  the  vertical  limits  of  the 
displayed  pathway. 

Percent  Time  in  Pathway,  Horisontal:  Percentage  of  time 
the  flight  path  was  flown  within  the  horizontal  limits  of 
the  displayed  pathway. 

Percent  Time  in  Both:  Percentage  of  time  the  flight  path 
was  flown  within  the  entry  gate  of  the  displayed  pathway. 

Exposure  Score:  The  integrated  time  ownship  was  within  the 
envelope  of  each  threat,  weighted  by  the  relative  lethality 
of  each  threat.  The  relative  lethality  of  a  threat  is  a 
function  of  type  (AAA  or  SAN)  and  the  state  of  that  threat 
(prebriefed,  search,  track,  or  launch).  Exposure 
score  reflects  the  pilots'  ability  to  deviate  from  the 
pathway  as  required  to  avoid  threats. 

Latencies  of  the  required  responses  were  derived  from  the  data 
acquisition  program  which  generated  a  detailed  record  of  mission 
events.  The  time  at  which  each  event  occurred  and  the  time  at 
which  the  crew  responded  were  recorded  to  produce  a  latency 
report.  For  example,  availability  of  an  alternate  route  or  a 
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Table  7.3-1.  Performance  Measures 


Dependent  Variables 

Condition 

LLP 

SH 

BVR 

RMS  Error,  Vortical 

a 

a 

RMS  Error,  Lattral 

a 

e 

%  Tima  in  Pathway,  Vortical 

a 

a 

H  Tima  in  Pathway,  Lattral 

a 

a 

HTimoinFath 

a 

a 

Exposure  Score 

e 

Latoncy  of  Routt  Saloction 

a 

a 

Correctness  of  Routt  Stitction 

a 

e 

Lattney  of  Climb  Profilt  Section 

a 

Correctness  of  Climb  Profile  Selection 

a 

Latoncy  of  Countermeasures  Release 

e 

f' 

a 

Latency  of  Response  to  Pop-Up  Threats 

a 

e 

Correctness  of  Pop-Up  Threat  Identification 

a 

a 

Latency  of  Damage  Report 

a 

Correctness  of  Damage  Report 

a 

Latency  of  Response  to  Target  Assignments 

a 

Correctness  of  Reponse  to  Target  Assignments 

a 

Latency  of  Response  to  Shoot  Cue 

_ 

a 
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change  in  threat  state  was  recorded  as  well  as  crew  input  in 
response  to  mission  events,  i.e.,  the  selection  of  an  alternate 
route  or  the  release  of  expendable  countermeasures.  Latencies  of 
the  required  responses  were  defined  as  follows. 

Latency  of  Route  Selection:  Elapsed  time  from  the 
presentation  of  the  alternate  route  to  the  navigation 
update  command. 

Latency  of  Climb  Profile  Selection:  Elapsed  time  from  the 
presentation  of  alternate  climb  profiles  to  the  designation 
of  the  start  of  climb  point. 

Latency  of  Countermeasures  Release:  Elapsed  time  from 
presentation  of  a  threat  launch  cue  within  the  display 
formats  to  the  control  input  releasing  the  countermeasures. 

Latency  of  Response  to  Pop-Up  Threats:  Elapsed  time  from 
the  appearance  of  a  pop-up  threat  within  the  display 
formats  to  the  aircrews*  verbal  report  of  that  threat. 

Latency  of  Damage  Report:  Elapsed  time  from  the  selection 
of  the  appropriate  status  switch  to  aircrews*  verbal 
report. 

Latency  of  Response  to  Target  Assignments:  Elapsed  time 
from  the  display  of  target  assignments  to  the  targeting  of 
weapons . 

Latency  of  Response  to  Shoot  Cue:  Elapsed  time  from  the 
presentation  of  the  shoot  cue  to  the  actuation  of  the 
trigger. 

Frequency  of  response  error  was  obtained  from  two  sources.  The 
required  verbal  responses  -  the  aircrews  climb  profile  selection, 
identification  of  pop-up  threats  (type  and  state),  and  the  status 
reports  of  degraded  aircraft  systems  -  were  obtained  from  the 
audio  channel  of  the  video  tape  flight  records.  The  computer 
generated  switch  history  indicated  whether  or  not  the  optimal 
route  alternatives  were  selected  and  the  appropricte  weapons 
targeted. 

A  multivariate  analysis  of  variance  (MANOVA)  was  performed  to 
determine  if  display  mode  effected  aircrew  preformanoe.  The 
independent  variable  was  display  presentation  mode  with  two 
levels  -  color  or  monochrome.  For  the  low  level  penetration 
mission  segments  the  dependent  measures  included  exposure  score, 
latency  of  route  selection,  latency  of  countermeasures  release, 
and  latency  of  response  to  pop-up  threats.  The  analysis, 
summarized  in  Table  7.3-2,  indicated  no  significant  difference 
between  the  two  display  presentation  modes  for  the  set  of  four 
performance  measures. 
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Table  7.3-2.  One-Factor  Repeated  Measures  MANOVA 


MANOVA  for  Lew  level  Penetration 

Multivariate  tests  of  significance  (S  •  1,  M  a  i  .0,  N  «  2.5) 


Test  name 

Value 

F 

Hypothesis 

DF 

Error  DF 

Significance 

ofF 

Pillais 

WmEM 

3.72  (approx.) 

1 

9 

0.0746 

Hotel  lings 

ESI 

3.72  (approx.) 

1 

9 

0.0746 

Wilks 

.217$ 

3.72  (exact) 

1 

9 

0.0746 

Roys 

2.477 

3.72  (upper  bound) 

_ 1 _ 

_ 9 

0.0746 

MANOVA  for  System  Health 

Multivariate  tests  of  significance  (Ssi.Ms  3.5,  N  *  1 

.5) 

Testnarr* 

Value 

F 

Hypothesis 

DF 

Error  DF 

Significance 

ofF 

Pillais 

.4879 

0.42  (approx.) 

1 

12 

0.8692 

Hotel  lings 

.9529 

0.42  (approx.) 

1 

12 

0.8692 

Wilks 

.5121 

0.42  (exact) 

1 

12 

0.8692 

Roys 

.9529 

0.42  (upp*r  bound) 

1 

12 

0.8692 

MANOVA  Beyond  Visual  Range 

Multivariate  tests  of  significance  (S=  1,  M  =  3.5,  N  =  2.5) 


Test  name 

Value 

F 

Hypothesis 

DF 

Error  DF 

Significance 

ofF 

Pillais 

.3517 

0.36  (approx.) 

1 

14 

■ 

Hotel  lings 

.5425 

0.36  (approx.) 

1 

14 

Wilks 

.6483 

0.36  (exact) 

1 

14 

mgm 

Roys 

.5425 

0.36  (upper  bound) 

1 

14 

■HI 
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MSS  error  data#  percentage  of  tine  within  the  pathway#  latency  of 
rotate  selection#  latency  of  countermeasures  release,  latency  of 
response  to  pop-up  threats#  and  latency  of  damage  report  were  the 
dependent  measures  used  to  execute  the  KAMOVA  for  the  system 
health  aission  segments.  The  summary  data  (Table  7.3-2)  revealed 
to  significant  performance  difference  between  color  and 
moncH&romo  displays. 

Ih*  beyond  visual  range  dependent  variables  included:  RMS  error 
det*#  percentage  of  time  within  the  pathway,  latency  of  climb 
profile  selection#  latency  of  countermeasures  release#  latency 
of  response  to  target  assignments#  and  latency  of  response  to 
shoot  cue.  The  KAMOVA  (Table  7,3-2)  failed  to  indicate  a 
significant  performance  difference  between  the  two  presentation 
modes. 

7.3.1  Low  Level  Penetration  Performance 

Exposure  score  and  accuracy  of  flight  within  the  displayed 
pathway  and  its  entry  gate  were  mutually  exclusive  flight  path 
control  measures.  Therefore,  in  low  level  penetration  segments# 
th«  exposure  score  reflected  the  pilots'  ability  to  deviate  fron 
tho  pathway  as  required  to  avoid  threats,  with  the  color  display 
suite#  mean  exposure  score  was  399.31;  with  the  monochrome 
display  suite#  mean  exposure  score  was  389.50. 

Latency  data  for  the  required  responses  indicated  no  significant 
differences  between  the  color  or  monochrome  display  presentation 
mode.  As  summarised  in  Table  7.3-3#  mean  response  time  for  the 
release  of  countermeasures  and  selection  of  an  alternate  rout* 
and  the  standard  deviations  associated  with  the  measurements 
generally  remained  consistent  between  display  modes.  Howevvr# 
in  the  monochrome  condition  slightly  longer  latencies  together 
with  substantial  standard  deviations  were  observed  in  the 
response  to  pop-up  threats. 


Table  7.3-3.  LLP,  Latency  of  Required  Responses 


Mean 

Standard 

Deviation 

N 

[  Latency  of  Route  Selection  j 

Color 

5.33 

2.71 

42 

Monochrome 

4.64 

2.41 

42 

Latency  of  Countermeasures  Release 

Color 

2.15 

0.82 

27 

Monochrome 

2.17 

0.72 

23 

Latency  of  Response  to  Pop-Up  Threats 

Color 

4.51 

4.68 

41 

Monochrome 

5.71 

5.00 

34 
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Frequency  of  error  data  were  examined  with  respect  to  alternate 
route  selection  and  identification  of  pep-up  threats  and  their 
state  (search,  track,  or  launch).  Summarised  in  Table  7.3-4,  the 
data  reveal  no  appreciable  difference  in  performance  between  the 
monochrome  and  color  displays. 


Table  7.3-4.  LLP,  Frequency  of  Error 


Correct  Incorrect  Default  N 

Alternate  Route  Selection 

Color 

Monochrome 

N 

30  12  11  53 

27  16  8  51 

57  28  19  104 

Response  to  Pop-Ups 
-Identification  of  Threat 

Color 

Monochrome 

N 

36  2  9  47 

32  1  14  47 

68  3  23  94 

Response  to  Pop-Ups 
-Identification  of  State 

Color 

Monochrome 

N 

MM 

While  the  data  reported  above  proved  inconclusive  in  establishing 
a  color/monochrome  performance  difference,  the  frequency  of  error 
data  was  useful  in  highlighting  weak  format  areas  common  to  the 
two  display  types.  Of  the  pop-up  threats  that  occurred  across 
all  low  level  penetration  mission  segments,  aircrews  failed  to 
report  twenty-four  percent  of  those  occasions.  However, 
failures  to  report  an  identified  pop-up's  state  were  much  higher 
-  sixty-five  percent.  A  decision  to  improve  the  distinction 
between  threat  states  may  reduce  this  high  frequency  of  error. 

7.3.2  System  Health  Performance 

In  the  system  health  segments,  aircrews  were  briefed  to  fly  the 
displayed  pathway  within  the  entry  gate  regardless  of  the  threat 
bedown.  The  flight  path  control  measures  of  interest  are 
summarised  in  Table  7.3-5. 
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Table  7.3-5.  SH,  Flight  Path  Control  Data 


Color 

Mono 

RMS  Error,  Vertical 

477.25 

953.94 

RMS  Error,  Lateral 

£39.14 

1420.19 

H  Time  in  Pathway,  Vertical 

77.13 

73.25 

HTime  in  Pathway,  Lateral 

69.88 

MTM 

%  Time  in  Roth 

65.19 

59.75 

Reported  in  Table  7.3-6  are  the  aircrew  response  latencies  in  the 
system  health  segments.  Included  are  latencies  for  route 
selection*  countermeasures  release*  response  to  pop-up  threats* 
and  latency  of  damage  report.  The  data  fail  to  indicate  any 
performance  difference  as  a  function  of  display  mode. 


Table  7.3-6.  SH,  Latency  of  Required  Responses 


Mean 

Standard 

Deviation 

N 

Latency  of  Route  Selection  I 

Color 

6.28 

336 

54 

Monochrome 

6.02 

2.92 

55 

Latency  of  Countermeasures  Release 

Color 

2.45 

1.13 

83 

Monochrome 

230 

1.55 

86 

Latency  of  Response  to  Pop-Up  Threats 

Color 

3.00 

2  58 

54 

Monochrome 

3.68 

5.41 

47 

Latency  of  Damage  Report 

Color 

7.58 

3.49 

92 

Monchrome 

8.25 

3.65 

84 

Frequency  of  error,  summarised  in  Table  7.3-7*  reveals  no 
substantial  difference  in  performance  between  the  monochrome  and 
color  displays.  The  high  rate  of  failure  (forty-two  percent)  to 
report  the  state  of  a  pop-up  threat  is  notewortly,  again 
indicating  the  necessity  for  improving  the  distinction  between 
states. 


TsMt  7.3-7.  $H,  Frtqutncy  of  Error 


Correct  Incorrect  Default  N- 

Alternate  Route  Selection 

Color 

MOnOUNUIItV 

N 

44  18  9  61 

34  It  6  58 

78  36  IS  119 

Response  to  Pop-ups 
•Identification  of  Threat 

Color 

Monochrome 

N 

49  2  13  64 

46  2  15  63 

95  4  28  127 

Response  to  Pop-Ups 
•Identification  of  State 

Color 

Monochrome 

N 

26  6  30  64 

25  IS  23  63 

S3  21  53  127 

Report  of  Damage 

Color 

Monochrome 

N 

£*2 

ooo 

Sf5S 

The  pilot's  identification  and  brief  description  of  the  current 
Malfunction  or  failure  was  useful  in  determining  if  an  aircraft's 
subsystem  could  be  displayed  pictorially  and  interpreted 
correctly.  The  performance  data  provides  no  strong  evidence  for 
the  modification  of  the  generic  system  formats  represented  in  the 
simulation. 

7.3.3  Beyond  Visual  Range  Performance 

The  flight  path  control  performance  data  generated  at  the 
conclusion  of  the  BVR  test  segments  is  summarized  in  Table  7.3-8. 
Latencies  of  climb  profile  selection,  countermeasures  release , 
response  to  the  shoot  cue,  and  response  to  the  target  assignments 
were  recorded  in  the  BVR  engagements  and  are  reported  in  Table 
7.3-9.  The  frequency  of  error  data  (summarized  in  Table  7.3-10) 
was  calculated  for  response  to  the  shoot  cue  and  target 
assignments. 
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Ta 6/t  7.M  ®Wt  W/ght  Path  Control  Data 


Color 

Mono 

iUQftMgM 

nMv  I  Wj  vvi  umN 

1944.09 

1359.44 

RMSIrror,  lotorol 

1492.91 

1955.19 

^  1  mW 111  TmuCV 

59.7S 

92.19 

WTtmo  In  Potiwmy.  Lotoroi 

•2.75 

wtsnsM 

%  TknoinOoth 

49.12 

53  44 

Tabh  7J-9.  §VKl*tftyofPaquir*dPaspons*s 


Moon 

Stondorti 

Deviation 

N 

lUrtoncy  of  Climb  ProtilaSoloction  j 

Color 

42.03 

10.13 

19 

|  HPiCTWlllWIW 

43.54 

9.74 

19 

1  1  ab«oou  ^mmAowbooiiyLO  bAlaxa  I 

I  UOIvIvy  Of  UNnHnnfCIUrVI  flPVIN  | 

C’tlor 

2.37 

1.19 

43 

|  Monochromo 

2.41 

1.51 

44 

IlMoncy  of  ftoaponw  to  Shoot  Cuo 

Color 

9.95 

9.43 

31 

Monochromo 

7.77 

11.43 

31 

1  Lotontyof  Umpon— toTorgotAMlonmcnti 

Color 

19.47 

1127 
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Monochonno 

22.54 

21.91 

**  J 

Tabia  7.3-10 l  §VK  Fntquancy  of  Error 


Gonoct 


NoAnpontt  Combination 


in* 


rr 


Corroct 


CMmbProfMoSalaction 

Color 

wtcnocnfvVfvv 

N 


No  SWjponjo 


The  latency  and  error  iata  fall  to  Indicate  a  significant 
difference  between  the  color  and  monochrome  displays.  However, 
the  calculation  of  the  frequency  of  error  in  response  to  targets 
assignments  was  unique.  Examination  of  the  detailed  record  of 
mission  events  and  crew  input  revealed  that  while  a  few  target 
assignments  were  incorrectly  executed  and  a  number  omitted 
entirely,  the  vast  majority  of  errors  were  the  result  of 
extraneous  assignments  in  addition  to  the  those  displayed.  The 
extraneous  assignments  are  referred  to  as  combination  errors. 

The  data  indicate  that  as  the  mission  progresses  and  the  display 
complexity  increases,  errors  increase.  The  example  shown  in 
Table  7.3-11,  drawn  from  the  record  of  mission  events,  serves  to 
illustrate  the  two  common  combination  errors.  The  failure  to 
readily  differentiate  between  ownship  and  wingman  target 
assignment  coding  and  to  differentiate  between  target  assignment 
coding  and  weapon  release  coding  may  be  due  to  the  use  of 
closely  related  symbology. 

Table  7.3 - 1 1.  BVR,  Target  Assignment  Response  Example 


Displayed  target 
assignment 

Crew  input 

Latency 

Error 

1.  Initial  assignments 

Target  A/C  3  and  4 

Target  A/C  3 
Target  A/C  4 

Ssec. 

5  sec. 

None 

None 

2.  Reassignment: 

Target  A/C  1 

— 

—  <N 

!K 

♦W  +4 

II 

27  sec. 

71  sec. 

None 

Extraneous  weapon 
assigned  to  a  wingman 
designated  target 

3.  Secondary  assignment: 
Target  A/C  3  and  6 

Target  A/C  3 
Target  A/C6 
Target  A/C  1 

•  sec. 

11  sec. 

79  sec. 

None 

None 

Extraneous  weapon 
assigned  to  an  A/C  coded 
with  Weapon  Release' 
symbology 

7.3.4  Repeated  HANOVAs 

The  logic  of  multivariate  analysis  of  variance  allows  for  use  of 
a  correlation  matrix  to  reduce  the  number  of  defendent  variables 
to  be  examined.  The  reasoning  is  that  if  two  variables  have  a 
large  amount  of  common  variance,  the  overall  error  variance  will 
be  reduced  by  eliminating  one  of  the  variables  from  the  MANOVA. 
Three  correlation  matrices  were  computed,  one  for  each  of  the 
mission  types.  A  priori ,  it  was  decided  that  the  threshold 
correlation  of  0.7  would  be  applied.  Thus,  if  two  variables  have 
49  percent  or  more  common  variance,  one  of  them  would  be 
eliminated. 


V 

V 


1% 

> 
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Table  7.3-12  la  tha  correlation  matrix  for  tha  data  uaed  In  tha 
low  lava!  penetration  MANOVA.  Tha  intercorrelattons  ware  quite 
low,  none  higher  than  0.29  (8.4  percent  couon  variance),  to  no 
dependent  variable  waa  eliminates  by  this  process  and  the  MANOVA 
was  not  repeated. 


Tab/#  7.3-12.  LLP,  Correlation  Matrix 


Exposure  Score  (ES) 

1.00 

Pop-Up  Latency  (PL) 

-0.19  1.00 

Countermeasures  Latency  (CL) 

0.08  -0  19  1.00 

Route  Selection  Latency  (RL) 

-0.17  0.29  0.12  1.00 

ES  PL  CL  RL 

Table  7.3-13  is  the  correlation  matrix  for  the  data  used  in  the 
system  health  MANOVA.  Based  on  these  correlations,  three 
dependent  variables  were  eliminated  for  a  repeat  MANOVA.  The  two 
BUS  error  scores  were  correlated  0.89  (79  percent  common 
variance).  Lateral  RMS  error  was  eliminated  because  it  also 
correlated  higher  than  vertical  RMS  error  with  several  of  the 
other  variates.  The  correlations  among  the  three  percent-time- 
in-pathway  scores  were  all  above  0.9.  Percent-time-in-pathway 
(both)  was  retained  because  it  logically  contained  the  other  two. 

When  the  MANOVA  was  repeated  on  the  reduced  set  of  six  dependent 
variables,  F  moved  from  the  0.42  reported  in  Table  7.3-2  to  0.66 
and  and  the  significance  of  that  F  moved  from  0.8692  to  .6840. 

The  color  versus  monochrome  difference  was  still  not  significant. 

Table  7.3- 13.  SH,  Correlation  Matrix 


RMS  Error.  Vertical  (RMSV) 

1.00 

RMS  Error,  Lateral  (RMSL) 

0.89 

wrm 

Percent  Time  in  Pathway, 
Vertical  (PATHV) 

a 

Percent  Time  in  Pathway. 
Lateral  (PATHL) 

III 

•0.65 

0.92 

1.00 

Percent  Time  in  Pathway, 
Both  (PATHS) 

•0.56 

-0.62 

0.94 

0.99 

1.00 

Damage  Latency  (DL) 

wrm 

EO 

-0.11 

-0.13 

-0.13 

1.00 

Pop-Up  Latency  (PL) 

0.01 

0.11 

•0.14 

-0.13 

-0.13 

0.26 

1.00 

Countermeasures  Latency 
(Cl) 

-0.07 

-0.05 

0.03 

•0.03 

in 

HI 

m 

1.00 

ftoute  Selection  Latency 
(RL) 

m 

0.45 

KB 

1.00 

RMSV 

RMSL 

PATHV 

PATHL 

PATHS 

DL 

PL 

i  ± 

RL 

102 


Table  7.3-14  Is  the  correlation  matrix  for  tha  data  usad  in  tha 
beyond  visual  range  HANOVA.  Based  on  these  data,  the  same  three 
dependent  variables  were  eliminated  as  in  systea  health.  The  two 
MB  error  scores  were  correlated  0.81.  Lateral  MB  error  was 
eliminated  because  it  correlated  higher  than  vertical  MB  error 
with  several  of  the  other  variates.  Again  tha  three  percent- 
time-in-pathway  scores  correlated  highly  (all  above  0.88);  and 
again  per cent- time- in-pathway  (both)  was  retained. 

When  the  HANOVA  was  repeated  on  the  reduced  set  of  six  dependent 
variables,  r  moved  from  0.36  reported  in  Tabic  7.3-2  to  0.65  and 
the  significance  of  that  r  moved  from  .9172  to  .6901.  Again  the 
color  versus  monochrome  difference  remained  non-significant. 

Tab/e  7.3- 14.  BVR,  Correlation  Matrix 


RMS  Error.  Vertical  (RMSV) 

1.00 

RMS  Error.  Latere!  (RMSl) 

0.81 

1.00 

Percent  Time  in  Pathway. 
Vertical  (PA1HV) 

1.00 

Percent  Time  in  Pathway. 
Lateral  (PATHL) 

m 

•0.68 

0.88 

1.00 

Percent  Time  in  Pathway. 
Both  (PATHS) 

•0.38 

•0.61 

092 

0.97 

1.00 

Climb  Profile  Latency  (CPI) 

•0.24 

-012 

0.14 

0.01 

1.00 

Countermeasures  Latency 
(Cl) 

m 

020 

•0.41 

•0.29 

•0.33 

0.07 

1.00 

Shoot  Latency  (51) 

0.23 

0.49 

-0.61 

-0.51 

-0.S3 

-0.14 

0.01 

1.00 

Target  Assignment  Latency 
(TL) 

m 

0.26 

•0.22 

•0.23 

-0.21 

m 

a 

RMSV 

|  RMSL 

WkTHV 

RATHL 

PATHS 

CM. 

CL 

LjlL 

TL 
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8.0  CONCLUSIONS  AND  FORMAT  REVISIONS 


The  intent  of  this  program  series  has  bssn  to  inprovs  information 
flow  to  tha  aircrew  by  use  of  pictorial  formats  and  to  evaluate 
the  extensive  use  of  color  formats.  The  program  has  established 
extremes  in  both  of  these  areas  -  pictorial  formats  and  color 
usage,  from  recorded  and  unrecorded  aircrew  comments ,  it  appears 
appropriate  to  retain  both  color  and  pictorial  formats,  but  to 
move  away  from  the  established  extremes,  words  and  numbers 
should  be  used  rather  than  symbolic  or  pictorial  representation 
where  they  lead  to  more  rapid  and  certain  information  flow. 
Similarly,  color  should  be  used  with  somewhat  more  restraint,  to 
highlight  particularly  important  information  or  to  indicate 
relations  across  displays,  but  not  to  **se  color  for  its  own  sake. 
Taken  together,  these  constraints  would  reduce  apparent  display 
clutter  and  increase  aircrew  awareness  of  critical  information. 
Seising,  Senyuh,  and  Martin  (1986)  concluded  that  MA  well 
constructed  color  pictorial  format  can  include  a  number  of  coding 
strategies  and  yet  concisely  provide  the  essential  data  required 
to  manage  the  aircraft  and  mission."  The  results  of  this  study 
support  this  conclusion. 

8.1  Usability  and  Acceptability  of  the  Formats 

The  first  objective  of  this  program  was  to  determine  if  the 
pictorial  formats  were  usable  and  acceptable  to  two  scat  fighter 
crews.  The  objective  was  met  and  the  clear  answer  is  that  the 
crews  found  these  formats  quite  usable  and  acceptable.  The 
aircrews  provided  significant  feedback  on  particular  formats. 

This  information  was  addressed  in  the  last  section  and,  where 
consensus  appeared,  format  revisions  are  suggested  in  paragraph 
8.3. 

The  process  of  designing  pictorial  formats  was  addressed  by 
Bdwards,  Way  and  Hornsby  (1986).  While  this  process  is  not 
unique,  it  is  an  effort  which  should  be  undertaken  whenever 
displays  are  designed.  The  design  process  bears  repeating  here 
because  a  complete  display  suite  was  designed  and  evaluated  in 
an  environment  free  from  some  of  the  constraints  of  previous 
display  efforts.  For  one  thing,  electronic  displays  provide 
clean  slates  upon  which  formats  can  be  developed  without  the 
problems  encountered  in  electromechanical  instruments.  For 
another,  working  with  a  complete  display  suite  allows  much  more 
emphasis  to  be  placed  on  compatibility  among  displays  than  if 
they  were  developed  one  at  a  time. 


The  display  development  process  began  with  a  preliminary 
definition  of  information  requirements  and  then  development  of  a 
point-of-departure  display  suite.  This  preliminary  suite  was 
then  evaluated,  refined,  and  evaluated  again  by  operational  air 
crews  -  all  in  the  form  of  static  pictures.  Then  a  systems 
engineering  exercise  .■> " s  undertaken  by  a  team  of  human  factors 
specialists,  other  engineers,  and  in-house  pilots.  A  mission 
scenario  was  defined  and  decomposed  to  yield  specific 
information-? ction  requirements.  Then  the  display  requirements 
were  assigned  to  elements  of  the  preliminary  display  suite  and 
static  formats  were  drawn  up  for  critical  events  during  the 
scenario  and  laid  out  in  an  event-by-event  and  format-by-format 
storyboard.  This  cockpit  wide  analysis  of  the  formats  was 
useful  in  highlighting  formatting  inconsistencies  among  and 
within  the  formats  and  in  identifying  changes  in  information 
requirements  at  different  points  through  the  mission. 

After  the  static  development  was  complete,  the  formats  were 
submitted  for  programming  and  integration  into  the  dynamic 
simulation.  Finally,  they  were  evaluated  twice  more  in  crew-in- 
the-loop  simulation  with  an  operationally  realistic  mission. 

8.2  Color  versus  Monochrome  Formats 

The  second  objective  of  the  program  was  to  develop  and  evaluate 
color  and  monochrome  versions  of  each  format.  The  results  agree 
with  many  others,  reviewed  by  Christ  (1975)  and  Silverstein 
(1982),  that  whether  or  not  color  aided  performance  in  an 
experimental  task,  it  was  preferred  by  the  subjects.  The  pilots 
and  WSOs  in  this  program  chose  the  color  versions  of  these 
formats  over  the  monochrome  in  almost  every  case. 

Assuming  that  color  is  used,  how  should  it  be  applied  in 
electronic  aircraft  cockpit  displays?  The  literature  provides  a 
number  of  lists  of  color  applications.  Krebs  and  Wolf  (1979) 
give  four  uses  for  color  in  information  displays;  (1)  as  an  aid 
in  locating  a  specific  symbol  in  a  clustered  or  information  dense 
display;  (2)  as  a  cue  or  alerting  signal  to  warn  or  inform  an 
operator  of  change  in  some  critical  parameter;  (3)  as  a  method  of 
grouping  similar  items  or  separating  dissimilar  items;  and  (4)  as 
a  method  of  increasing  visibility  of  some  information  item  by 
adding  color  contract  to  brightness  contrast. 

Krebs  and  Wolf  (1979)  also  warned  that  Color  may  be 
inappropriately  used  and  may  distract  the  operator  or  reduce  the 
effective  rate  of  information  flow.  Examples  of  inappropriate 
use  of  color  include:  (1)  color  noted  by  the  operator  but  which 
has  no  task  oriented  meaning;  (7.)  symbols  of  the  same  color 
which  are  inappropriately  grouped  by  the  operator;  (3)  over  use 
of  color  which  interferes  with  its  attention  getting  value. 
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8.3  Recommended  Format  Revisions 


The  third  program  objective  was  to  use  results  from  the 
simulation  to  recommend  revision  to  the  formats,  where 
appropriate.  Suggestions  are  made  here  for  revisions  to  the  HUD, 
the  PSF,  the  HSF ,  the  Close  Look  Format  and  the  Engine  Status 
format.  No  revisions  are  recommended  to  the  other  status 
formats,  the  advisory  formats,  or  the  programming  formats.  The 
aircrews  found  these  acceptable  and  usable  and  they  did  not,  with 
any  consensus,  suggest  revision.  These  formats,  in  particular, 
are  sensitive  to  particular  applications  and  will  reflect  the 
systems  and  the  missions  of  specific  aircraft. 

8.3.1  Head-Up  Display 

As  the  primary  flight  display,  the  pictorial  format  of  the  Head- 
Up  Display  proved  to  be  adequate  in  most  areas.  It  is  suggested 
that  the  terrain  and  ground  plane  be  retained  in  their  present 
form.  Readouts  of  airspeed,  heading,  and  altitude  could  be  used 
more  effectively  in  a  pilot's  scan  pattern  if  positioned  closer 
to  the  center  of  the  display.  Adding  a  heading  tape  to  the  top 
of  the  display  would  facilitate  heading  change  manuevers. 

Aircrews  were  also  critical  of  the  roll  indicator  and  index, 
arbitarily  limited  to  45°.  It  is  recommended  that  the  roll 
indicator  and  index  be  replaced  with  a  horizontal  reference  line 
as  shown  in  Figure  8.3-1.  This  line  would  provide  the  pilot  with 
an  immediate  and  intuitive  indication  of  roll  and  limited  pitch. 
The  horizontal  reference  line  indicates  the  aircraft's 
orientation  relative  to  the  earth  at  any  attitude.  At  zero  roll 
and  zero  pitch,  the  horizontal  reference  line  appears  as  a  pair 
of  solid  lines  extending  from  either  side  of  the  ownship  symbol. 
For  pitch  up  the  feet  of  the  solid  horizontal  refesence  line 
point  down  toward  the  horizon  and  for  pitch  down  the  feet  of  the 
dashed  line  point  up  toward  the  horizon.  The  artificial 
reference  line  overlays  the  horizon  when  it  is  in  the  fit  Id  of 
view.  At  more  extreme  pitch  angles,  the  horizontal  reference 
line  pegs  at  the  top  (for  pitch  down)  or  bottom  (for  pitch  up), 
thereby  continuing  to  provide  roll  information  with  an  immediate 
indication  of  direction  back  to  horizontal  flight. 

The  minor  adjustments  suggested  above  are  intended  to  improve  the 
utility  of  the  pathway.  The  segmented  white  pathway  symbology 
appears  to  be  acceptable  as  does  the  symbology  of  its  associated 
elements  -  the  pathway  entry  gate  (flight  director)  and  the 
ownship  symbol  (velocity  vector).  The  conspicuousness  of  the 
entry  gate  and  ownship  symbol  would  be  improved  with  heavier 
lines.  The  pathway  was  frequently  found  to  be  inadequate  as  a 
steering  device  and  it  was  extremely  difficult  to  recover  using 


106 


rvxajAz ruma  aaiusj.iu  /■\_»  *v*»  i ’.apjimpl*  pl*  a* r«  »v*  , 


360  f  027 ! 


Figure  8.3-1.  Revised  Head-Up  Display  -  Air  Mode 


the  transitional  flight  director.  The  pathway  in  its  current 
state  of  development  appears  to  ke  best  suited  as  a  predictive 
element  to  be  used  when  the  pilot  is  adherring  to  a  flight 
profile  or  when  pathway  guidance  back  to  that  flight  profile  can 
be  generated. 

The  pitch  ladder  would  not  generally  be  used  when  on  a 
predetermined  flight  profile.  Yet  in  an  air-to-air  engagement  or 
when  executing  threat  avoidance  maneuvers,  it  may  not  be  possible 
or  desirable  to  generate  or  display  a  pathway.  In  such  cases  the 
pitch  ladder  would  be  selected  to  replace  the  pathway.  Later, 
when  the  pilot  wished  to  return  to  the  original  flight  profile, 
the  pathway  would  be  reselected  to  replace  the  pitch  ladder.  The 
pathway  guiding  the  aircraft  to  the  original  flight  profile  may 
be  differentially  color  coded  until  the  return  is  complete. 

Using  a  pathway  to  return  to  aircraft  to  the  original  flight 
profile  would  eliminate  the  requirement  for  the  clumsy 
transitional  flight  director. 

in  addition,  £-  pilots  have  insisted,  the  HUD  should  include  the 
capability  to  display  an  alternate  pathway  constructed  with 
waypoints  inserted  by  the  aircrew  using  the  HSF.  In  essence,  the 
revised  HUD  seeks  to  explore  mechanisms  for  implementing  a 
pathway  at  differing  levels  such  that  the  presence  or  absence  of 
a  pathway  best  suits  a  particular  phase  of  flight. 

Other  minor  modifications  are  suggested  for  the  attack  HLE  and 
shoot  cue.  Throughout  the  display  suite,  the  color  coding  green 
when  used  in  conjunction  with  the  targeting  of  a  weapon  has 
indicated  the  within  range  status  of  the  weapon,  except  for  the 
attack  MLiL.  By  simply  reversing  the  white  and  green  areas  along 
the  arrow,  consistency  across  displays  is  acheived  with  green 
within  range  coding  and  white  no-escape  coding.  The  shoot  cue, 
an  X  within  the  ownship  symbol,  was  difficult  to  see  and  to  many 
pilots  it  represents  a  break  away  indication.  It  is  suggested 
that  the  word  SHOOT  replace  the  X  as  a  shoot  cue. 

8.3.2  Perspective  Situation  Format 

Modifications  to  the  basic  symbology  of  the  Perspective  Situation 
Format  are  minimal.  A  predictive  flight  vector  has  been  attached 
to  the  ownship  symbol  to  provide  aircraft  direction  information 
based  on  current  attitude.  However  a  frequently  cited  problem 
was  that  the  lock-on  circle  around  ownship  obscured  other 
critical  symbology.  Transparency  of  the  disk  would  be  an  ideal 
solution.  However,  without  such  a  graphics  capability,  a  lock-on 
ring  would  be  sufficent  to  indicate  the  track  condition.  The 
track  and  launch  conditions  could  be  further  differentiated  with 
an  amber  tractor  beam  and  lock-on  ring  indicating  a  tracking 
threat  and  a  red  tractor  beam  and  lock-on  ring  indicating  a 
launching  threat. 


Figure  6.3-2  is  a  revision  of  the  PSF  in  air  node.  It  is 
intended  to  provide  the  aircrew  with  a  quick  look  of  the  air 
battle  situation,  details  of  which  are  provided  by  the  Close  look 
and  Horisontal  Situation  Formats.  in  *-his  revision,  the 
perspective  space  is  shown  as  a  segment  of  a  right  circular 
cylinder  bounded  at  the  bottom  by  the  ground  plane  and  at  the  top 
by  a  horizontal  plane  through  ownship' s  altitude.  The  radius  of 
the  cylinder  and  angular  width  of  the  segment  are  functions  of 
ownship's  radar  coverage.  The  ground  plane  is  ruled  off  in 
range  arcs  and  ten  degree  relative  bearing  radials.  The  height 
cf  the  segment  can  be  inferred  from  ten  thousand  foot  tick  narks 
along  the  left  and  right  sides  of  the  radar  coverage  area. 

Within  the  space  thus  defined,  ownship  is  on  the  axis  of  the 
solid  segment  atop  a  stake  running  down  to  the  origin  of  the 
figure.  Other  aircraft  in  the  space  are  on  stakes  whose  lengths 
represent  altitude  and  origins  on  the  ground  plane  represent 
range  and  bearing.  Radar  coverage  sectors  are  attached  to  the 
nose  of  an  aircraft  in  search  node.  Airborne  threat's  track  and 
launch  symbology  is  identical  to  that  of  surface  threats. 

8.3.3  Horizontal  Situation  Format 

The  HSF  remains  essentially  unchanged  with  only  minor  detailing 
in  order  to  provide  more  precise  information  in  selected  areas. 

As  with  the  PSF,  a  predictive  flight  vector  has  been  attached  to 
the  ownship  symbol. 

Aircrews  frequently  critised  the  range  ring  implementation  where 
the  rings  always  represented  ranges  of  one  quarter  and  one  half 
of  the  selected  format  range,  but  the  ring  interval  was  not 
explicit.  Range  may  be  better  implemented  as  a  series  of  rings 
where  range  is  increased  in  specified  steps  fron  ownship,  the 
size  being  that  which  best  fits  the  display  range  in  useful 
multiples.  The  recommended  revision  is  to  display  ring  interval 
with  the  selected  format  range,  when  range  rings  have  been 
selected.  Recommended  range/ring  interval  combinations  are: 

20/5,  40/10,  80/20,  100/25,  150/25  and  300/50.  Note  that  the 
available  format  ranges  have  also  been  changed  to  provide  another 
intermediate  range. 

In  Figure  8.3-3,  a  crew  selectable  compass  rose  has  been  added  to 
the  Horizontal  Situation  Format's  heading  readout.  When  both  the 
range  rings  and  the  compass  rose  are  selected,  the  compass  rose 
forms  the  outermost  range  ring.  The  compass  rose  and  the 
optional  range  rings  are  available  to  provide  qualitative 
pictorial  information  concerning  the  global  sitiuation.  However, 
as  the  tactical  situation  develops,  precise  bearing  and  range 
information  may  be  required.  The  cursor  can  be  used  to  designate 
a  given  location  for  display  of  the  numeric  bearing  and  range. 
Similarly,  time  and  distance  to  a  particular  point  may  be  better 
implemented  as  a  cursor  function. 
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Figure  8.3-2.  Revised  Perspective  Situation  Format  -  Air  Mode 
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Figure  8.3-3.  Revised  Horizontal  Situation  Format  -  Ground  Mode 


One  contributor  to  clutter  on  the  HSF  nay  bo  the  use  of 
abbreviated  symbology  for  threats.  In  air  mode,  ground  threats 
were  shown  as  iewna  without  the  threat  envelopes.  The  rugg*st.«d 
change  is  to  remove  the  icons  and  show  only  air  threats  in  air 
mode  and  ground  threats  in  ground  mode.  The  all-threats  feature 
would  be  retained  to  display,  at  crew  option,  both  air  and  ground 
threat  envelopes  in  either  mode.  Other  declutter  features  are 
possible  and  could  be  added,  depending  on  aircraft  ana  mission 
application. 


As  with  the  PSF,  the  solid  lock-on  circle  was  reduced  to  a  long- 
on  ring  to  avoid  obsuring  other  symbology.  To  better  distinguish 
between  the  track  and  launch  conditions,  an  amber  tractor  beam 
and  lock-on  ring  indicates  a  tracking  threat;  a  red  tractor  beam 
and  lock-on  ring  indicates  a  launching  threat. 

In  air  mode,  correlation  between  the  HSF  and  CLF  was  weak  with 
respect  to  aircraft  identification  and  missile  launch  envelope 
depiction.  From  observation  and  informal  discussion  with  the 
aircrews,  execution  of  an  engagement  with  two  or  more  formations 
was  difficult.  To  coordinate  the  symbology  of  the  two  displays, 
squares  and  circles  with  attached  flight  vectors  should  represent 
unknown  and  friendly  aircraft  or  formations  of  aircraft  on  the 
HSF  as  they  do  on  the  CLF.  To  assist  the  aircrew  in  effectively 
using  the  HSF  (in  conjunction  with  the  CLF),  identification  and 
access  to  each  formation  was  simplified.  Once  selected  for  Close 
Look  display  an  aircraft  symbol  would  be  enclosed  by  a  set  of 
four  box  corners  and  designated  with  an  identifying  letter.  The 
identifying  letter  is  then  associated  with  one  of  the  switches 
alongside  the  display.  As  subsequent  formations  are  selected  for 
Close  Look  display,  sets  of  box  corners  and  identification 
letters  are  added  to  each  aircraft  or  formation  of  aircraft  and 
to  additional  side  switches.  As  shown  in  Figure  8.3-4  color 
coding  may  be  used  to  differentiate  that  formation  currently 
displayed  on  the  CLF  from  those  formations  stored  in  the  track 
file  as  a  result  of  their  initial  designation.  The  identified 
side  switches  are  then  used  to  select  the  formation  to  be 
aisplayed  on  the  CLF.  Target  assignment  symbology  will  also  be 
added  to  an  HSF  symbol  as  they  appear  within  the  formation  should 
it  be  displayed  on  the  CLF.  An  advantage  to  displaying  a  target 
assignment  ring  within  the  HSF  is  that  it  cues  the  pilot  to 
select  the  appropriate  formation  on  the  CLF  for  targeting  of 
weapons . 

Figure  8.3-5  illustrates  the  threat  and  ownship  symbology  changes 
OSr  the  HSF  in  air  mode.  Few  changes  are  recommended.  The 
tractor  beams  which  represent  missile  launch  envelopes  of  threats 
and  ownship  appear  as  elongated  "L"  shapes  and  are  more 
prominent.  Graphics  rules  will  preclude  overlaying  of  the 
tractor  beams.  The  order  of  colors  on  ownship' s  tractor  beam  has 
been  reversed  to  match  the  HUD  and  the  stores  status  format. 

Now,  within  range  is  green  and  within  the  no-escape  zone  is 
white. 
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Figure  8.3-4.  Revised  Horizontal  Situation  Format  -  Air  Mode 


8,3,4  Clot*  Look  Format 

Significant  changes  ara  recommended  for  tha  CLP  to  simplify  its 
use  with  tha  BSP.  Tha  uae  of  two  Cloaa  Look  Formats,  each  with  a 
subset  of  the  necessary  information,  required  excessive  display 
management  and  occasioned  significant  confusion.  It.  is 
recommended  that  a  single  foraat  be  used  which  would  display 
complete  formations,  selected  from  the  HSP,  as  before.  The  CLP 
would  center  about  the  centroid  of  the  formation  (not  necessarily 
a  particular  aircraft)  and  have  one  of  four  crew  selectable 
ranges:  5na,  lOna,  2Cn»,  and  30nm.  The  sise  and  number  of 

ranges  allows  for  an  entire  formation  to  be  contained  within  a 
single  format.  The  formation  centroid  is  marked  with  a  small 
white  cross  and  is  the  point  to  which  range  and  bearing  are 
calculated. 

The  symbology  identifying  a  multiple  group  of  aircraft  and  those 
aircraft  determined  to  be  friendly  and  unknown  remains  unchanged. 
However,  the  diamond  previously  used  in  coding  hostile  aircraft 
is  reduced  to  a  triangle  (as  in  the  HSP).  The  three  point  scale 
identifying  aircraft  as  hostile  (red  triangle),  unknown  (amber 
square),  and  friendly  (green  square)  is  sufficient,  so  the 
interpolated  "probable  hostile"  and  "probable  friendly"  symbols 
have  been  eliminated.  The  simplification  is  expected  to  reduce 
confusion.  The  flight  vector  is  always  attached  to  the  nose  of 
the  aircraft  symbol.  A  modification  of  the  weapon  status  coding 
was  required  to  assist  the  aircrew  in  distinguishing  between 
target  assignment  coding  (blue  rings  tor  ownship)  and  weapon 
release  coding  (thin  green  rings  for  ownship  and  wingman).  Once 
a  weapon  has  been  fired,  the  target  aircraft  symbol  and  its 
coding  are  reduced  in  sise. 

As  the  Identification  Friend,  Poe,  or  Neutral  process  identifies 
aircraft  type  and  model,  that  information  is  displayed  within  the 
symbol  as  previously.  An  identification  number  is  included 
within  each  individual  symbol;  the  same  identification  number  is 
associated  with  a  side  switch  for  targeting.  Readouts 
indicating  heading  (degrees),  airspeed  (knots),  and  altitude  (in 
thousands  of  feet)  for  each  aircraft  are  added  to  the  display 
alongside  the  identified  switches  (Figure  8.3-6).  An  optional 
declutter  feature  is  available  to  remove  the  aircraft  information 
readouts  on  the  sides  of  the  format  (but  retain  the  aircraft 
identification  number). 

To  map  the  formation  selected  from  the  HSP  (indicated  with  a  set 
of  differentially  coded  box  corners)  for  CLP  display,  a  readout 
identifying  the  formation  was  added  to  the  lower  edge  of  the  CLP 
display.  The  revised  CLF  also  provides  a  mechanism  to  display 
aircraft  in  the  event  a  formation  breaks  up  and  forms  two 
distinct  tactical  groups.  As  the  formation  differentiates  into 
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Figure  8.3-6.  Revised  Close  Look  Format 


two#  tho  addition  of  a  now  foraation  occur*  automatically  within 
tho  track  filo.  Tho  idontif lcatlon  o £  tho  additional  formation 
nem«rt  in  tho  RSFt  tho  noxt  availablo  twitch  it  identified  and 
the  appropriate  tymbol  la  encloaed  within  a  now  tot  of  box 
corners. 

8.3.5  8n9ine  Statue  Format 

Tho  eoncopta  depicted  in  tho  Bngine  Statue  Format  wore  well 
racoivod  and  worked  well  for  tho  pilote.  The  notion  of  shoving 
compoeite  thruat  directly#  rather  than  tho  uaual  thruat 
corrolatoa  auch  aa  Nl#  N2#  or  bps  waa  well  accepted.  The 
criticiama  and  suggested  roviaiona  wore  in  tho  direction  of 
aimplification  and  "display  by  exception."  Figure  8.3-7  ahowa 
the  recommended  Bngine  Status  Format  roviaiona.  The  information 
between  the  two  engine  bodies  has  been  removed  and  the  engine 
bodies  moved  closer  together  to  facilitate  cross  checks.  The 
labels  have  been  deleted  and  the  numeric  fuel  flow  readout  has 
been  movod  into  the  fuel  flow  arrows.  The  engine  oil  pressure 
and  temperature  indications  and  BGT  have  been  retained  but  will 
be  displayed  when  needed#  rather  than  full  time.  The  resulting 
Bngine  Strtus  Format  retains  the  same  information  elements  as 
before  but  is  simpler#  responsive  to  pilot  feedback. 
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APPENDIX  A 


PILOT  RESPONSES  TO  STATIC  FORMAT  EVALUATION  QUESTIONNAIRES 

The  Static  Format  Evaluation  is  discussed  in  Section  2  of  this 
report.  Responses  to  the  open-ended  questions  used  in  the  static 
format  evaluation  are  summarized  in  this  appendix.  Parenthetical 
numbers  indicate  frequency  of  the  comment. 


Head  Up  Display 

Information  to  be  added : 

Information  to  be  deleted: 

Comments: 

Pitch  ladder  (2) 

PITS  fill  (3) 

OK  for  air-to-ground;  of  little  or 
doubtful  use  for  air-to-air  (2) 

Heading  scale  (2) 

PITS,  in  A/A  mode  (1) 

Makes  PITS  fill  optional  to  avoid 

Reciprocal  heading  at  bottom  (1) 

1000-ft.  markers  on  PITS  (1) 

obscuring  view  (2) 

Target  designator  (1 ) 

N" 

Keep  pitch  ladder  (1) 

Weapon  status,  when  Master 

Terrain  outlines  good,  but  have 

arm  on  (1) 

declutter  switch  to  delete 
unnecessary  information  (1) 

!  Range  and  ti  me-to-go  to  target 

|  (D 

i 

Have  a  standard  ADI  for  primary 
flight  instrument  (1) 

[  Bomb  impact  point  (1) 

Format  good  for  navigation  (1 ) 

|  A/A  and  A/G  symbology  ( 1 ) 

f 

[  Missile  Launch  Envelope  Symbology  (HUD)  1 

[  Information  to  be  added : 

* 

Information  to  be  deleted: 

Comments:  1 

1  Target  designator  box  (3) 

l 

Flight  path  (3)V 

Don't  need  pathway;  keep 
simpler  (2) 

Steering  cue  (3) 

Terrain  outlines  (1) 

Not  bad,  but  prefer  symbology  in 

Target  type,  range  (2) 

F-15(better  than  F- 1 6)  ( 1 ) 

Weapons  selected  (2) 

Airspeed,  not  mach  (1) 

Weapons  remaining  (1) 

1 

Shoot  cue  (1) 

Pitch  ladder  (1) 

| 

Missil  i  launch  parameters;  cues 

i 

to  meet  them  (1) 

| 

A-2 

\  'I 

SS5122AJ/131-7 

PSF,  Ground  Mode 


Attitude  information  (2) 
Pitch  information  (2) 

*DI(1) 

Adjustable  range  scale  ( 1 ) 
Specific  probable  threats  ( 1 ) 

Information  to  be  added : 

Ceding  for  highest  priority 
threat  -  red  or  flashing  (3) 

Range  to  threats  (2) 

Threat  altitude  (2) 

Pitch  ladder  (1) 

Ground  threat  envelopes  ( 1 ) 


Information  to  be  added : 

Enemy  altitude  (2) 

Specific  threat  labels  (1) 

Cardinal  heading  cues  0) 

Written  launch  warning  (1) 

Option  to  display  A/A  threat 
envelopes,  “friendlies*  (1) 


Information  to  be  deleted:  Comments: 

(No  responses)  Good  (3) 

In  monochrome,  lock-on  beam  can 
be  confused  with  grid  lines  (2) 

Need  larger  lock-on  beam  for  active 
threats 


PSF,  Air  Mode 


Information  to  be  deleted:  Comments: 

Target  altitude  arrows  ( 1 )  Display  absol  ute  altitude,  not 

relative  altitude,  for  enemies  (1) 

Flip  relative  altitude  arrows.  (1 ) 

Not  too  useful  (1) 

Difficult  to  interpret  range  and 
aspect  of  enemies  ( 1 ) 

“New  View*  good  for  backseater 
(1) 


HSF,  Ground  Mode 

Information  to  be  deleted :  Comments: 

(No  responses)  "New  View*  seems  useful,  but  for 

WSO.  (4) 

"New  View”  capability  extremely 
favorable  for  survivability.  (1) 

Good  display  (1) 

Veiy  busy  to  took  out  if  fast  and  low. 

0) 


A-3 


SS5122/V/131-7 


HSF,  Air  Mode 


infannitiamJa  nfetat 

Range  to  target 
Type  of  mi  nil#  launched  (2) 
Airspee d,  altitude  of  aircraft  ( 1 ) 
Numbers  of  aircraft  ( 1 ) 

Cardinal  heading  (1) 

Ground  threat  status.  (1) 


Information  to  be  added : 

Altitude  information  (3) 
Selected  vertical  information  (1) 

information  to  be  added: 

Overall  failure  picture;  what  is 
lost  by  failure  (1) 

None(1) 


Information  to  be  deleted:  Comments: 

(No  responses)  Display  too  complicated/cluttered 

(2) 

Good  concept  (1) 

Display  absolute  altitude,  not 
relative  altitude,  for  enemies  ( 1 ) 

Delete  threat  envelopes  for 
threats  that  aren't  pointed  at  us 
(1) 

?Ncw  View"  quite  useful  for  A/G 
(1) 

Move  airmode  to  HUD  ( 1 ) 


Target  Formation  Display 


Information  to  be  deleted: 

Diamond  symbology.  Just 
aircraft  type  (1) 

Electrical  Status 

Information  to  be  deleted: 

Status  of  all  normal 
electrical  components  (1) 

None(1) 

Everything  but  problem 
identifier  (1) 


Comments: 

Useful  (2) 

Incorporate  into  HSF  as  range  to 
target  decreases  ( 1 ) 


Comments: 

Not  more  useful  than  current 
telelight  panel  (4) 

May  lose  information  if  there  are 
problems  with  more  than  one 
system  (2) 

Display  prioritized  checklist  (2) 

Prefer  system  lost/problem,  then 
list  what  you  would  lose  (1) 


i 

A-4 


SSS122/W/131-7 


Electrical  Status  (Continued) 


Electrical  Advisory 


lls«  red  only  for  critical  action 
problems;  flashing  amber  for 
non-critical  failures  (1) 


Information  to  be  delete 


Comments: 


None(1) 


Unnecessary  wosds  (1 ) 
None(1) 


Very  good  (2) 

Good,  but  not  necessary  (2) 

Useful  with  telelight  panel  (2) 

Have  pilot  call  this  up  on  same  CRT 
as  Electrical  Status  (1) 


Hydraulic  Status 


None(1) 


The  whole  format  (1) 

Everything  but  failed 
system  ID  (1) 


Comments: 

List  degraded  or  failed  systems  (4) 

Easy  to  interpret  (1) 

Pictures  easier  than  readi  ng  ( 1 ) 

Don’t  need  (2) 

Limit  information  to 
operation-critical  (1) 

Like  amber  for  potential  problems 

(D 

Show  normal  systems  in  green  (1) 

Flash  symbology  for  new  failures 

(D 


$$5123/0/131-7 


Fuel  Status 


intanHrtigfmhridrt; 

Fuel  quantity  in  each  tank  (4) 

Total  fuai  gauga,  constantly 
aval  labia '3) 

Wing  fual  balance  gauge  (1) 
Numeric  readouts  (1) 

(No  responses) 


Information  to  be  deleted:  Comments: 

(No  responses)  A  constant  sou  rce  of  total  fuel 

information  is  required  (3) 

Prefer  fuel  gauge  (3) 

Need  more  information  about  fuel 
in  each  tank(1) 

Has  good  potential  as  a 
supplement  to  fuel  gauge  (1) 

Stores  Status 


Information  to  be  deleted:  Comments: 

"Master  Arm  Off"  except  if  Display  it  automatically  whenever 
format  is  brought  up  Master  Arm  is  on  (4) 

manually  (1) 

Display  it  all  the  time  (1) 

Display  it  automatically  whenever 
a  weapon  status  changes  ( 1 ) 

Nice  display  (1) 

Not  much  better  than  current  ACO 
(1) 

Need  capability  to  look  at  4  or  5 
things  with  pressing  buttons  or 
using  voice  commands  to  change 
displays  (1) 
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Countermeasures 


Information  to  bo  added: 

Actual  number  of  chaff  and 
flares  (1) 


Information  to  be  added : 

Move  ownship  symbol  in  globe 
to  show  current  attitude  (1) 


Information  to  be  deleted:  Comments: 


(No  responses)  OK  for  WSO  (2) 

Only  need  to  know  if  something  is 
wrong  (2) 

Prefer  number  of  chaff/flares  left 
(1) 

Only  need  low  level  warning  light 
(1) 

ECM  "lightning  bolts"  not  required 
unless  jamming  is  directional  (1) 

Would  be  used  only  prior  to  FEBA 
(1) 


Passive  Sensor  Status 

Information  to  be  deleted:  Comments: 

(No  responses)  Display  it  only  upon  failure  or 

when  called  up  (2) 

Would  seldom  be  used  (2) 

Good  for  WSO  (2) 

Only  for  pre-FEBA  or  when  time 
available  (1) 

Integrate  with  other  displays  (1) 
Should  be  ownship  relative  (1) 
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Engine  Status 


Sannaotti 


Numeric  readouts  (4)  Carets  for  suggested  Prefer  or  need  gauges/round  dials 

thrustAhrottl#  position  (1 )  (6) 

Fuel  flow  in  pounds  per  hour  (2) 

Actual  versus  selected  thrust  Need  numeric  readouts  (4) 
is  not  required  (t) 

Must  be  displayed  at  all  times  (2) 

Afterburner  flames,  fuel 

flow  arrows(l)  Prefer  warnings  or  caution 

information,  plus  gauges  (2) 

Include  hydraulics  (1 ) 

Oil  display  is  excellent  (1) 


Engine  Advisory 


Cgrnyntflttl 


(No  responses}  (No  responses)  Good,  useful  (3) 

Like  checklist  formats  (1) 

Use  round  dials  with  warning  lights 
and  be  able  to  call  up  this(1) 

Need  both  engine  display  and  this 
at  the  same  time  (1) 

Monochrome  reduces  distraction 
(1) 


A-8 
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APPENDIX  B 


AIRCREW  QUESTIONNAIRES 

MEAN  USABILITY  RATINGS  OP  FORMAT  SYMBOLOGY  ELEMENTS 

This  appendix  contains  usability  profiles  of  the  formats  and 
their  symbology  elements.  These  profiles  were  derived  by 
plotting  the  mean  usability  ratings  given  by  the  aircrews  in  the 
questionnaires.  The  mean  ratings  for  usability  and  other 
attributes  are  given  in  Appendix  C. 


B-l 


The  symbology  element  is:  < 


VERY 

JS22SL 


V«rv  Good 
Moderately  Good 
Slightly  Good 
Neutral 
Slightly  boor 
Moderately  Poor 
Very  Poor 


^  With  . 


aspect  to  usability 


VERY 

POOR 


inm  tUdIMl 

VMM#  IWnlflV 


Pathway -Filled 


Pathway -Outline 

Pitch  Ladder 

Transitional  Flight  Director 

Zero  Pitch  Reference  Line 
« 

Threat  Alert  Symbols 

Threat  Summary  Symbols  (number  of) 

Missile  or  AAm  Azimuth  Vector 

Time  to  Missile  Impact  Readout 

Heading  Readout 

Airspeed  Readout 

Attitude  Readout 

Heading.  Mach.  Altitude  Command 
Arrows  and  Readouts 

Roll  Index 

Waypoints 

velocity  Vector 


*  *  COLOR 

O- - -O  MONOCHROME 

Usability  Ratings  of  HUD  Symbology  Ekmtnts  In  Ground  Mock  (Maan  of  16  Pilots) 


B-2 


SSSOS7/A/327-E 


Tha  symbology  alamant  It:  c 


1 


V«ry  Good 
Mooarataly  Good 
Slightly  Good 
Nautral 
Slightly  Poor 
Modarataly  Poor 
Vary  Poor 


>  With  ratpact  to  usability 


VERY  ^  VERY 


Usability  Kttlngs  of  HUD  Symbology  Ehments  In  Air  Mods  (Moon  of  16  Pilots) 


SSS057/B/327-6 


Tha  symbology  element  li:  <  4. 


Kvl'J 


»•»  INfnWn 


Ownship  Symbol 


Planned  Flight  Route 


Ground  Grid 


Terrain  Altitude  Coding 


Airspeed  Readout 


Haading  Raadout 


Altitudo  Readout 


Ground  Throats: 


Throat  Lothality  Volume 


Track  Mode  Tractor  Boom 


Launch  Mode  Tractor  Beam 


Missile  Symbol 


Lock  On  Circle  (around  ownship) 


Airborne  Threat  Symbol 


New  View  Readouts 


Ownship  Preview  Symbol 


Very  Good 
Moderately  Good 
Slightly  Good 
Neutral 
Slightly  Poor 
Moderately  Poor 
Very  Poor 


With  respect  to  usability 


•  *  COLOR 

0- - -Q  MONOCHROME 


Usability  Ratings  of  PSF  Symbology  In  Ground  Mode  (Mean  of  16  Pilots  and  16  WSOs) 


SS5057/C/327-6 


VERY 

GOOD 


PSF  Symbology  Element 


Ownship  Symbol 


Planned  Flight  Route 


Terrain  Altitude  Coding 


Ground  Grid 


Groundpoint  Synbol 


Mach  Readout 


Heading  Readout 


Altitude  Readout 


Airborne  Threats: 


Threat  Symbols 


Radar  Coverage  Sectors 


Track  Mode  Tractor  8eam 


Launch  Mode  Tractor  Beam 


Missile  Symbol 


Lock  On  Circle 


SAM  Missile  Symbol 


AAA  Gun  Symbol 


Tracked  A/C  Readou . 


closing  rate 


relative  altitude 


Ownship  Preview  Symbol 


1.  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  <  4.  Neutral  >  With  respect  to  usability 

5.  Slightiy  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


// 


4 

4 

4 

4 

V 

>  4 

-  4 


COLOR 


0. - .Q  MONOCHROME 


Usability  Ratings  of  PSF  Symbology  In  Air  Mode  (Mean  of  16  Pilots  and  16  WSOs) 


SS5057/D/327-6 


B-5 


t .  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  <  4.  Neutral 

5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


VERY 

GOOD 


With  respect  to  usability 


SS5057/E/327-6 


1 .  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  <  4.  Neutral 

|  5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


VERY 

GOOD 


HSP  Symbology  Element 


Ownship  Symbol 


Planned  Flight  Route 


Display  Range  Rings 


Fuel  Range  Rings 


Airborne  Threats: 


Aircraft  Symbols 


Threat  Radar  Coverage  Sector 


Track  Mode  Tractor  Beam 


Launch  Mode  Tractor  Beam 


MLE  Boundary  Arcs 


Missile  Location  and  Type 


Ownship  Radar  Coverage  Area 


Ownship  Tractor  Beam 


MLE  Boundary  Arcs 


Ownship  Missile  Symbol 


SAM  Missile  Symbol 


AAA  Gun  Symbol 


Waypoint  and  Target  Symbols 


Display  Range  Readout 


V>  ith  respect  to  usability 


•  •  COLOR 

© - .©  MONOCHROME 


Usability  Ratings  of  HSF  in  Air  Mode  (Mean  of  16  Pilots  and  16  WSOs) 
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r 


The  symbology  element  is: 


< 


1. 

2. 

3. 

A 

5 

6. 

7. 


Very  Good 
Moderately  Good 
Slightly  Good 
Neutral 
Slighv'ly  Poor 
Moderately  Poor 
Very  Poor 


With  respect  to  usability 


VERY  V  VERY 

GOOD _ POOR 


Detail  CLF  Symbology  Element 

— 

: 

— 

i  i 

— 1  ■ 1 — 

l  ! 

i  l 

- 1 

i  7 

Aircraft  Classification  Symbols 

/ 

Aircraft  Type  Alphanumerics 

4$ 

Aircraft  Flight  Vectors 

Aircraft  Relative  Altitude  Readouts 

Aircraft  Mach  Readouts 

Target  Recommendation  Coding 

9 

t 

Weapon  Status  Coding 

1 

i 

Aircraft  ID  Numbers 

Usability  Ratings  of  Close  Look  Formats  (Mean  of  16  Pilots  and  16  WSOs) 


SS5057/G/327-6 


B-8 


J1.  Vary  Good 

2.  Modarataly  Good 

3.  Slightly  Good 

4.  Nautral 

S.  Slightly  Poor 

6.  Modarataly  Poor 

7.  Vary  Poor 


VERY 

GOOD 


With  raspact  to  usability 


•  *  COLOR 

0. - 0  MONOCHROME 

Usability  Ratings  of  Stores  Status,  Stores  Programming,  Countermeasures 
Status  and  Countermeasures  Programming  Formats 
(Mean  of  IS  Pilots  and  16  WSOs) 


SSS0S7/H/343-6 


Symbology  Ikuwm 


1 
2 

The  symbology  •lament  is:  ^  4 

6 

7 


VERY 

_52^e_ 


Very  Good 
Moderately  Good 
Slightly  Good 
Neutral 
Slightly  Poor 
Moderately  Poor 
Very  Poor 


>  With  respect  to  usability 


J 


VERY 

POOR 


1  2  3  4  5  6  7 


Recommended  Thrust  Carets 


Thrust  Handle  Bar 


Actual  Thrust  Bar 


Mil  Power  and  Afterburner  'coding* 


Fuel  Flow  Arrows 


Oil  Pressure  Status  Indicators 


Oil  Quantity  Status  Indicators 


EGT  Status  Indicators 


- - -  COLOR 

0- - -Q  MONOCHROME 


Usability  Ratings  of  Engine  Status  Format  (Mean  of  16  Pilots) 


t 

! 


SSS057/V327-6 


B-10 


1  Very  Good 

2  Moderately  Good 

3  Slightly  Good 

The  symbology  element  is:  c  4  Neutral 

5  Slightly  Poor 

6  Moderately  Poor 
7.  Very  Poor 


VERY 

GOOD 


Fuel  Status  Symbology  Elements  1 


Fuel  Quantity 


With  respect  to  usability 


*  COLOR 

0. - -Q  MONOCHROME 


Usability  Ratings  of  Fuel  Status.  Electrical  Status,  and  Hydraulic  Status  Formats 
(Mean  of  16  Pilots) 


SS505//J/009-7 


1.  V try  Good 

1 2.  Moderately  Good 
3.  Slightly  Good 

The  symbology  element  it:  <  4.  Neutral 

^  5.  Slightly  Poor 
S.  Moderately  Poor 
7.  Very  Poor 


>  With  respect  to  usability 


VERY  V  VERY 


System  Advisory  Features 

Advisory  Title  and  Level 
(cautionAwarning) 


Action  Item  Arrow  i  Color  Coding 


Completed  Item  Check  &  Color  Coding 


\ 

d 


9  •  COLOR 

G" - <D  MONOCHROME 

Usabi'.ty  datings  of  Passive  Sensor  Status  (16  Pilots)  and  System  Advisory  Formats  (16  Pilots) 


SS50S7/K/062  7 
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APPENDIX  C 


AIRCREW  QUESTIONNAIRES 

MISSION  SEGMENT  AND  COMPOSITE  MISSIONS 

At  the  completion  of  the  test  flights  for  each  of  the  Mission 
segments,  the  aircrews  completed  a  questionnaire  on  the  formats 
which  were  featured  in  that  segment.  The  pilots  completed  one 
each  time  and  the  NSOs  completed  a  questionnaire  after  the  low 
level  penetration  and  BVR  air-to-air  segments,  in  addition,  both 
pilots  and  NSOs  completed  a  broader  questionnaire  on  information 
interpretation  after  the  final  full  mission  flight. 

Appendix  C  contains  in  pages  C-2  through  C-17  mean  pilot 
usability  ratings  of  the  display  symbology  for  each  of  the  three 
tust  segments.  NSO  mean  usability  ratings  are  reported  in  pages 
in  C-25  through  C-34.  Information  interpretation  questionnaires 
and  responses  are  given  in  pages  C-18  through  C-24  for  pilots  and 
C-35  through  C-38  for  NSOs.  The  NSOs  had  no  HUD  and  were  not 
responsible  for  system  health  responses  so  they  were  not 
questioned  on  the  HUD  or  most  of  the  system  status  formats. 


Ol 


PILOTS'  RESPONSES  TO 


SYSTEM  HEALTH  SEGMENT  QUESTIONS 

The  purpose  of  this  questionnaire  is  to  elicit  your  opinions  of 
the  symbology  used  in  this  study  Defined  below  are  the 
attributes  or  characteristics  that  you  will  rate  for  each  type  of 
display  symbology. 

Usability  in  Color.  How  easy  was  it  to  make  use  of  this 
display  element  in  the  color  display  mode? 

Consplcuousness  in  Color.  How  easy  was  it  to  see  this 
display  element  in  the  color  display  mode? 

Usability  in  Monochrome.  How  easy  was  it  to  make  use  of 
this  display  element  in  the  monochrome  display  mode? 

Conspicuousness  in  Monochrome.  How  easy  was  it  to  see  this 
display  element  in  the  monochrome  display  mode? 

Location,  is  this  format  element  in  the  right  place  and  on 
the  right  display? 

Meaning.  How  clear  or  obvious  is  the  meaning  of  uhis  format 

element? 

Precision.  Does  this  format  element  convey  its  information 
with  the  appropriate  level  of  precision? 

Timeliness.  Is  this  format  element  available  to  you  at  the 
right  time  and  for  the  right  duration? 

Training.  How  easily  could  this  format  element  be  learned? 

Workload.  Does  this  format  element  contribute  to  workload 
or  relieve  it? 
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The  symbology  element  is: 


1. 

2. 

3. 

4. 

5. 

6. 

7. 


Very  Good 
Moderately  Good 
Slightly  Good 
Neutral 
Slightly  Poor 
Moderately  Poor 
Very  Poor 


>  with  respect  to  this  attribute 


In 

Color 

In 

Mono-  Mode-independent 

chrome  Attributes 

A 

Display:  Engine  Status  /  /c5 

n 

¥ 

Recommended  Thrust  Carets 

2.1 

2.1 

2  5 

B 

2.3 

g 

ia 

g 

2.2 

Thrust  Handle  Bar 

2.0 

m 

m 

2.2 

F 

m 

g 

28 

23 

Actual  Thrust  Bar 

1.8 

1.9 

1.8 

m 

ea 

19 

*•*1 

2  3 

18 

1.9 

Mil  Power  and  Afterburner  "coding" 

2.6 

2.9 

3.1 

m 

2.5 

2.5 

2.6 

2  3 

26 

Fuel  Flow  Arrows 

3.9 

3.9 

4.1 

m 

m 

m 

4.1 

39 

39 

3.5 

Oil  Pressure  Status  Indicators 

2.1 

B 

2  5 

m 

2  5 

m 

2.3 

2.6 

24 

2.5 

Oil  Quantity  Status  Indicators 

2.1 

2.3 

SI 

m 

2  5 

2.3 

2.3 

26 

2  3 

2  5 

EGT  Status  Indicators 

2.2 

m 

a 

3  3 

25 

2.3 

2.6 

26 

23 

23 

, 

Display:  Fuel  Status 


Fuel  Quantity 

1.8 

1.8 

1.9 

2  2 

2  3 

2.3 

3  3 

2  2 

26 

Pump  Symbols 

2.3 

2.2 

2.6 

si 

2.6 

28 

g 

g 

2  5 

28 

Failure  coding 

28 

2.6 

Bl 

3.6 

3.1 

3.1 

32 

3.1 

Damage  Coding 

2  5 

2  3 

3.1 

g 

2.9 

28 

g 

2  9 

28 

g 

Alternate  Fuel  Flow  Routes 

2  8 

EB 

3.1 

3.6 

3.1 

2  8 

B 

29 

29 

B 

SS50S2-K/308-6 


C-3 


The  symbology  element  is: 


1.  Very  Good 

2.  Moderate!; 

3.  Slightly  Go 

4.  Neutral 

5.  Slightly  Po< 

6.  Modtratel' 

7.  Very  Poor 


Display:  Electrical  Status 
Electrical  System  Elements 
Electrical  System  Buses 
Failed  Elements 


Failed  Buses 


Display:  Hydraulic  Status 

Aircraft  Elements _ 

Failed  Elements 

Single  Threat  Elements 

ID  of  Failed  Subsystem 

Caution/Warning  Borders 


SS5052-U308-6 


with  respect  to  this  attribute 


In 

In 

Mono- 

Color 

chrome 

Mode-independent 

Attributes 


1.8 

1.9 

1.6 

2.3 

1.9 

1.8 

1.6 

2.3 

1.9 

2.5 

2.2 

2  3 

1.9 

2.3 

2  2 

2.2 

2.1 


90! 

mm 

Stag 


2.8  2.4  3  2  37  28  2  9  3  3  1 3  0  13.0  |3.1 


2.3  2.3  3  7  4.3  2.9  3.1  3.1  2  9  3.1 


2.5  2.2  3.4  4.1  2.7  2.9  2.8  2.7  2.9  2.8 


2.4  2.3  3.1  38 


mm 

SBBE3SI 


2.9  2.6  3.4  3.6  2.9  2.6  2.9  2.8  2.8  2.8 
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1 .  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

Th«  symbology  is:  <(  *  N«*™|  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


Display:  Passive  Sensor  System  Status 


>  with  respect  to  this  attri  bute 


In 

In 

Mono- 

Color 

chrome 

Mode-independent 

Attributes 


System  Coverage  Globe 

2.3 

2.1 

0 

9 

9 

2  6 

2  8 

2.8 

Degraded  Sectors 

2.3 

2.6 

9 

9 

2.9 

9 

3.1 

29 

2.9 

3.1 

Ownship 

2.3 

2.3 

29 

9 

9 

9 

9 

2  9 

9 

28 

\ 


i 


Display:  Advisories 


Advisory  Title  andv  Level 
fcaution/warnina) 

1.7 

18 

2.3 

2.3 

2.2 

2.3 

2.2 

Action  Item  Arrow  &  Color  Coding 

1.8 

1.9 

2.3 

2.6 

19 

1.9 

2.1 

2.1 

1.9 

,9! 

Completed  Item  Check  &  Color  Coding 

1.8 

1.9 

2  2 

2.5 

19 

1.9 

9 

2.1 

SS5052-M/308-6 
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PILOTS'  RESPONSES  TO 


LON  LEVEL  PENETRATION  SEGMENT  QUESTIONS 

The  purpose  of  this  questionnaire  is  to  elicit  your  opinions  of 
the  symbology  used  in  this  study.  Defined  below  are  the 
attributes  or  characteristics  that  you  will  rate  for  each  type  of 
display  symbology. 

Usability  in  Color.  How  easy  was  it  to  make  use  of  this 
display  element  in  the  color  display  mode? 

Conspicuousness  in  Color.  How  easy  was  it  to  see  this 
display  element  in  the  color  display  mode? 

Usability  in  Monochrome.  How  easy  was  it  to  make  use  of 
this  display  element  in  the  monochrome  display  mode? 

Conspicuousness  in  Monochrome.  How  easy  was  it  to  see  this 
display  element  in  the  monochrome  display  mode? 

Location.  Is  this  format  element  in  the  right  place  and  on 
the  right  display? 

Meaning.  How  clear  or  obvious  is  the  meaning  of  this  format 
element? 

Precision.  Does  this  format  element  convey  its  information 
with  the  appropriate  level  of  precision? 

Timeliness.  Is  this  format  element  available  to  you  at  the 
right  time  and  for  the  right  duration? 

Training.  How  easily  could  this  format  element  be  learned? 

Workload.  Does  this  format  element  contribute  to  workload 
or  relieve  it? 


The  symbology  element  is: 


1 .  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

4.  Neutral 

5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


with  respect  to  this  attribute 


In 

In 

Mono- 

Color 

chrome 

- 7 - 3 

Mode-independent 

Attributes 


Display:  HUD  GROUND  MODE 
Display  Element: 


Pathway  -  Filled 


Pathway -Outline 


Pitch  Ladder 


Transitional  Flight  Director 


Zero  Pitch  Reference  Line 


Threat  Alert  Symbo's 


Threat  Summary  Symbols  (number  of) 


Missile  or  AAA  Azimuth  Vector 


Time  to  Missile  Impact  Readout 


Heading  Readout 


Airspeed  Readout 


Altitude  Readout 


Heading,  Mach,  Altitude  Command 
Arrows  and  Readouts 


Roll  Index 


Waypoints 


Ownship  Symbol 


fflr/mr/ 


2.8  3.1 


IBB 

IBB 


3.5 

3 

3.3 

3, 

2.3  1 2.6  1 2.4  2.3  2.4  2  6  |2.9  |2.6 


I 

1 


3.6 

3.3 

36 

3.5 

3.1 

3.5 

011310301099 


3.0  32  24  2.5  2  5  2.3  2  3  2.6 


1.6  |2.2  2  3  1.6  1.5  1.9  1.6  15  18 


2.1  2.5  28  24  2.1  2.2  2.1  2.1  2  3 


3.4  3.1  3.4  2.6  2.5  29  2.4  2.7  2.9 


3.3  3.4  3.4  2  4  2.4  2  6  2.3  2  3  3. 


2.2  2.3  2.4  2  0  2.1  2.2  1.9  1.8  2. 


IBBBB 

aBoBlonoBB 
bdoobdodI 
■hbbbbbbb 

■piDDBDDO 

■Bbbbbbbb 


2.5  2.4  2.6  2.6  21  2.1  2.0  2.1 


28  2.6  2.8  2.9  2.3  2.9  2.9  2.9  2.9  2.9 


3.3  3  6  3.4  3.6  3.4  3.0  3.1  2.5  3.2  3.2 


1.9  1.7  2.1  2.0  1.8  1.8  1.9  1.9  1.9  1.8 


2  1  2  4  2  2  2  3  1.8  2.3  2  4  2  0  1.9  2.0 


'z&oar*' 


1.  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  4  Neutral 

'  S.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


Display:  PSF  GROUND  MODE 
Display  Element: 


Ground  Threats: 


In 

In 

Mono- 

Color 

chrome 

Mode-independent 

Attributes 


Ownship  Symbol 

2.3 

2.8 

2.8 

16 

2.1 

18 

2.3 

2.1 

Planned  Flight  Route 

a 

2.1 

2.3 

28 

2.1 

1.9 

2.2 

2.0 

2.3 

2  4 

Ground  Grid 

2.5 

1.9 

a 

2  3 

18 

2.2 

a 

2.2 

2.2 

Terrain  Altitude  Coding 

2.8 

2.8 

a 

5.1 

2.9 

3.3 

4.1 

3.6 

34 

36 

Airspeed  Readout 

2.8 

2.8 

2.9 

2.9 

a 

2.5 

2.5 

2.3 

2  6 

2.6 

Heading  Readout 

3.1 

2.8 

3.1 

2.9 

2.6 

a 

26 

2.6 

2  8 

2  8 

Altitude  Readout 

9 

2.8 

3.1 

2.9 

2  9 

a 

2  6 

2.6 

28 

2  8 

Threat  Lethality  Volume 

1.5 

1.4 

3.1 

3.9 

1.5 

1.4 

1.4 

1.7 

2.0 

1.8 

Track  Mode  Tractor  Beam 

1.8 

1.9 

a 

2.9 

1.9 

1.5 

18 

18 

1.7 

Launch  Mode  Tractor  Beam 

1.9 

1.9 

0 

2  8 

2.1 

16 

1  8 

1.8 

18 

Missile  Symbol 

a 

2.1 

2.3 

2  6 

1.9 

1.9 

2.2 

2.1 

2.3 

Lock  On  Circle  (around  ownship) 

2.3 

1.9 

2.6 

2  6 

a 

2.3 

2  3 

2.1 

Airborne  Threat  Symbol 

2  9 

2  9 

0 

3  0 

0 

2  5 

2.5 

2.6 

2  8 

28 

New  View  Readouts 

32 

3.2 

36 

a 

2.9 

2  9 

29 

2  9 

9 

9 

Ownship  Preview  Symbol 

3.0 

3.1 

3  1 

3  2 

3.0 

9 

2  8 

2  8 

2  9 

9 

SS5052-H/308-6 


C-8 


t 


AAA  Lethality  Area 

1.3 

, 

1.1 

a 

3.3 

1.4 

1.3 

1.6 

1.7 

1.8 

1.8 

SAM  Lethality  Area 

1.2 

1.1 

a 

a 

1.4 

1.3 

1.6 

i.7 

1.8 

1.8 

Track  Mode  Tractor  Beam 

1.9 

1.9 

2.5 

2.9 

1  6 

a 

a 

1.9 

1.9 

2.2 

Launch  Mode  Tractor  Beam 

2.0 

2.1 

2  5 

2.9 

1.8 

18 

2.1 

1.9 

2.1 

2.3 

Missile  Location  and  Type 

2.0 

2.3 

2.3 

2.8 

1.8 

1.6 

2.1 

a 

a 

2.3 

Lock  On  Circle 

B 

2.1 

2.8 

2  5 

2  8 

2.0 

2.1 

2.5 

2.1 

2.6 

Airborne  Threat  Symbols 

a 

2.6 

3.3 

2.8 

2.8 

2.6 

2.6 

2.8 

a 

Ownship  Missile  Symbol 

32 

3.2 

3.5 

a 

a 

2.9 

2.9 

3.5 

3.2 

3.3 

FLOT 

2.6 

2  2 

2.8 

2  8 

“  j 

2.5 

2.5 

2.5 

a 

2.9 

General  Cursor  Symbol 

3.0 

3  6 

3  2 

a 

a 

36 

a 

Waypoints  Target  Symbol 

2.3 

2  5 

23 

"J 

2  3 

2  3 

2  3 

Display  Range  Readout 

2  6 

26 

2  6 

26 

2.7 

_ 

2.4 

2  1 

2  6 

2  6 

2.9 

c-c 


SS5052-I/308-6 


1 .  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  <  f  Neutral 

5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


with  respect  to  this  attribute 


Display:  Countermeasures  Status 


Jammer  Status 


Expendables  Symbology 


Expendables  Status 


in 

In 

Mono- 

Color 

chrome 

Mode-independent 

Attributes 


WWffWf 


2.5  |  2.0j  2.7  1 2.5  f  2.6  1 2.5  |2.3  |2.5  |2.4  |2  6 


2.1  2.1  2.1  2.5  2.4  2.3  2.3  2.5  2.3  2.5 


fwmm 


Display:  Countermeasures 
Programming 


Quantity  Selection  Options 


Release  Method  Options  (single, 
burst  salvo) 


Permission  Required  Selection 


1.6  |  1.7  1 1.6  1 1.7  1 1.8  1 1.6  1 1.9  |2.4~Tl.9  |2. 


1.7  1.6  1 1.7  1.5  1.9  2.1  1.7  2.1 


MBIBIBIBBlBffi 


2.1  2.0  2.1 


PILOTS'  RESPONSES  TO 


AIR  -  TO  -  AIR  BVR  SEGMENT  QUESTIONS 

The  purpose  of  this  questionnaire  is  to  elicit  your  opinions  of 
the  symbology  used  in  this  study.  Defined  below  are  the 
attributes  or  characteristics  that  you  will  rate  for  each  type  of 
displaiy  symbology. 

Usability  in  Color.  How  easy  was  it  to  make  use  of  this 
display  element  in  the  color  display  mode? 

Conspicuousness  in  Color.  How  easy  was  it  to  see  this 
display  eleaent  in  the  color  display  node? 

Usability  in  Monochrome.  How  easy  was  it  to  make  use  of 
this  display  element  in  the  monochrome  display  mode? 

Conspicuousness  in  Monochrome.  How  easy  was  it  to  see  this 
display  element  in  the  monochrome  display  mode? 

Location.  Is  this  format  element  in  the  right  place  and  on 
the  right  display? 

Meaning.  How  clear  or  obvious  is  the  meaning  of  this  format 
element? 

Precision.  Does  this  format  element  convey  its  information 
with  the  appropriate  level  of  precision? 

Timeliness.  Is  this  format  element  available  to  you  at  the 

right  time  and  for  the  right  duration? 

1- 

Training.  How  easily  could  this  format  element  be  learned? 

Workload.  Does  this  format  element  contribute  to  workload 
or  relieve  it? 


C-ll 


The  symbology  ol«m«nt  is: 


1.  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

4.  Neutral 

5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


Display:  HUD  AIR  MODE 
Display  Element: 


with  respect  to  this  attribute 


In 

In 

Mono- 

Color 

- 7 - 1 

chrome 

- 7 - 2 

Mode-independent 

Attributes 


WS 


Pathway  -  Filled 


Pathway  -  Outline 


Pitch  Ladder 

Transitional  Flight  Director 
Zero  Pitch  Reference 


DEFense  MLE  Arrow  and  Carets 
ATTack  MLE  Arrow  and  Carets 
Threat  Alert  Symbols 
Threat  Summary  Symbols  (number  of) 

Missile  Azimuth  Vector 
Time  to  Missile  Impact  Readout 


Vertical  Velocity  Indicator 


Heading  Readout 


Mach  Readout _ 

Altitude  Readout 

Heading,  Mach,  Altitude  Command 
Arrows  and  Readouts 


2.3  2.3  12.5  2.6  12.1  2.1  2.0  1.9  2. 


3.1  2.8  3.3  3  2  26  2.3  38  3.6  3  3 


3.4  3.1  1 3.8  3.3  2  2  2  2  4.0  3.5  3  2 


2.4  2.1  1 2.4  2  2  1  9  1  7  2.4  2  5  2. 


4.8  4.0  4.8  4.0  4  1  1 4.4  5.1  5.3 


2.3  2.1  2.4  2  5  2  2  2.1  2.3  2.3  2. 


1.9  1.9  2.3  2.6  2.0  1.8  2.0  2.0  1.8 


1.8  1.8  2.1  2  6  1  9  1.9  2.1  1.9  1.7 


1.8  1.9  1 1.9  2.1  1.7  1.6  1.9  1.8  1.9 


2.4  2.5  2.7  2.8  2  6  2.4  2.3  2.2  2.3 


1.9  2.1  2.1  2  3  1  9  1.8  2  1  1  8  1  8 


2.3  2.5  2.5  2  7  2.3  1  9  2.3  2.1  1  9 


3  5  3  3  4.5  3.8  3.9 


ID 

ID 

ID 


3.0  2.6  3  1  2.8  12  4  2.6  2.8  2.7  2.8 


2.5  1  9  26  2.1  2.1  24  2.1  2.1  2.3 


2.6  2  1  28  2.3  2.1  2.1  2.2  2.2  2.6 


2  4  2  3  2.6  2.6  1.9  2.2  2.6  2.2  2. 


SS50S2A/308-6 


The  symbology  element  is:  < 


1.  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

4.  Neutral 

5.  Slight'yPoor 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


Threat  Symbols 

: 

2.5 

2.6 

2.9 

3  3 

D 

D 

2.9 

22 

2.4 

2.8 

Radar  Coverage  Sectors 

2.1 

1.9 

2.5 

2.8 

22 

1.9 

2.5 

23 

2.2 

2.2 

Track  Mode  Tractor  Beam 

2.3 

2  2 

2  6 

B 

B 

2.1 

2.5 

B 

2.5 

2.6 

Launch  Mode  Tractor  Beam 

B3 

2.2 

2  3 

2  9 

2.1 

1.8 

2.1 

□ 

B 

2  3 

Missile  Symbol 

1.7 

1.8 

2.1 

2.3 

a 

1.5 

1.8 

1.7 

18 

m 

Lock  On  Circle 

1.9 

m 

2.6 

2.9 

2.2 

1.9 

B 

2.1 

B 

2.1 

SAM  Missile  Symbol 

m 

2.1 

a 

a 

2.3 

2.3 

2.3 

2.2 

2.1 

2.2 

AAA  Gun  Symbol 

2  5 

2  2 

2.5 

2  6 

2  3 

2.3 

2.3 

2  3 

2  1 

2.3 

Tracked  A/C  Readouts  range 

2.9 

m 

3.1 

3  1 

2  9 

7.8 

29 

2  9 

2  9 

closing  rate 

3  2 

3  2 

3  3 

3  3 

3  3 

3.1 

2.9 

31 

3  1 

3.1 

relative  altitude 

31 

m 

3  2 

3  0 

29 

2  9 

Ownship  Preview  Symbol 

EB 

36 

38 

3  6 

3  6 

3.6 

3.6 

36 

EB 

EB 

SS5052-B/308-6 


013 


1 .  Very  Good 

2 .  Moderate!  y  Good 

3.  Slightly  Good 

The  symbology  element  is:  <  }  Neutral 

'5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


Display:  HSFAIR  MODE 
Display  Element: 


In 

In 

Mono- 

Color 

chrome 

Mode-independent 

Attributes 


Ownship  Symbol 

1.5 

: - — 

1.5 

L-  .  ... 

2.3 

C - 

2.6 

1.7 

1.3 

1.6 

f 

1.6 

r  ■  , 

1.5 

r-  „ 

1.8 

Planned  Flight  Route 

1.6 

1.5 

m 

2.3 

1.7 

1.5 

1.9 

1.7 

1.7 

1.9 

Display  Range  Rings 

EB 

2.4 

2.6 

28 

m 

2  5 

2  5 

2  3 

2.5 

2.6 

Fuel  Range  Rings 

3.6 

3.1 

3.8 

EB 

28 

EB 

36 

29 

3.1 

3.3 

AirborneThreats: 


Aircraft  Symbols 

1.8 

2.2 

33 

1  8 

18 

2.1 

18 

1.9 

2.1 

Threat  Radar  Coverage  Sector 

1.9 

1.5 

2.1 

2  3 

1  8 

19 

1.9 

31 

Track  Mode  Tractor  Beam 

2.2 

m 

2.5 

29 

SI 

m 

1.9 

1.9 

2.2 

2.1 

Launch  Mode  Tractor  Beam 

2.2 

ED 

2.5 

29 

23 

23 

2.1 

m 

2.5 

2.5 

MIE  Boundary  Arcs 

2.8 

3.1 

3.3 

3.9 

2.6 

2.6 

SI 

2.3 

2.8 

29 

Missile  Location  and  Type 

2.3 

2.8 

SI 

3.1 

m 

2.3 

2.1 

m 

2.5 

3.1 

Ownship  Radar  Coverage  Area 

1.7 

1.6 

1.9 

19 

1.6 

1.5 

1.7 

1.7 

1.6 

1.9 

Ownship  Tractor  Beam 

2.8 

23 

3.1 

2.9 

2  5 

EB 

2.6 

2.6 

o 

28 

MLE  Boundary  Arcs 

2.8 

32 

3.3 

3.3 

26 

2.5 

26 

2.3 

m 

2.8 

Ownship  Missile  Symbol 

28 

3.1 

3.3 

2.9 

29 

29 

3  2 

3.3 

SAM  Missile  Symbol 

2  3 

2  3 

2.5 

ED 

EB 

23 

2.3 

2.2 

2.5 

2.2 

AAA  Gun  Symbol 

2  5 

2  5 

3.1 

2  5 

m 

24 

2  3 

SI 

I 

Waypoint  Target  Symbols 

2.0 

m 

m 

2  0 

19 

1.9 

2  3 

Display  Range  Readout 

2  1 

2  3 

E 

2.5 

0 

2  3 

1.9 

2.4 

SS5052-C/308-6 
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1.  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

Th* symbology *J*m*nt is:  ^ 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


In 

In 

Mono* 

Color 

chrome 

Mode-i  ndependent 
Attributes 


Display:  CLF  DETAIL  Version 


Aircraft  Classification  Symbols 

0 

1.9 

2.7 

3.0 

2.2 

2.7 

2.2 

2.1 

2.5 

2.5 

Aircraft  Type  Aiphanumerics 

1.8 

1.8 

1.9 

2.0 

2.1 

2.0 

1.9 

1.9 

1.8 

1.9 

Aircraft  Flight  Vectors 

9 

2.8 

2  7 

2.7 

29 

28 

2.7 

2.7 

27 

28 

Aircraft  Relative  Altitude  Readouts 

28 

2.9 

28 

29 

2.9 

28 

25 

26 

28 

3.2 

Aircraft  Mach  Readouts 

2.7 

2.8 

2.7 

28 

28 

2.7 

26 

2.8 

28 

3.2 

Target  Recommendation  Coding 

m 

1.9 

2.9 

3.7 

2.3 

28 

2.1 

2.2 

30 

3.2 

Weapon  Status  Coding 

2.1 

2.4 

2  5 

34 

2.3 

23 

2.1 

2.1 

29 

2.8 

Aircraft  10  Numbers 

2.2 

-■ 

1.7 

2.3 

1.9 

2.3 

24 

1.9 

1.9 

2.2 

2.3 

sssss2-onoe-6 
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1 .  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  ^  }  Neutml  - 

'5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


In 

In 

Mono- 

Color 

chrome 

Mode-i  ride  pendent 
Attributes 


Display:  CLF  FORMATION  Version 


Aircraft  Classification  Symbols 

1.9 

1.9 

1.9 

2.3 

1.9 

1.9 

1.6 

1.5 

1.6 

1.8 

Aircraft  Flight  Vectors 

2.1 

2.2 

2.1 

2.3 

1.8 

1.9 

2.3 

1.7 

22 

2.3 

Target  Recommendation  Coding 

1.8 

1.9 

2.4 

3.4 

1.7 

2.2 

1.8 

1.8 

2.0 

2.5 

Weapon  Status  Coding 

1.9 

2.0 

2.2 

2.8 

1.8 

2.1 

1.7 

18 

2.5 

2.5 

Aircraft  ID  Numbers 

2.1 

1.9 

2.1 

1.9 

2.0 

2.4 

1.8 

1.8 

1.9 

2.3 

Aircraft  Geometric  Arrangement 

1.7 

1.4 

1.7 

1  S 

1.6 

1  7 

16 

1.5 

18 

19 

Ownship  Bearing  Vector 

2.2 

2.3 

2.2 

24 

2.2 

2.2 

20 

1.9 

2.3 

2.2 

CLF  Range  Change  Feature 

EQ 

3.1 

3.0 

3.2 

3.0 

28 

2.7 

2.6 

3.2 

3.1 

SSS052-E/308-6 
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1. 

2. 


Very  Good 
Moderate* 


yGood 


3  Sllohtlv  Good 
Th*  i^nbology  ti:  Ng*™^ 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


Display:  Stores  Status 


In 

In 

Mono- 

Color 

chrome 

Mode-independent 

Attributes 


Weapon  Selected  Coding 

1.S 

1.6 

2.2 

Eg 

1.7 

1.9 

1.9 

1.7 

1.8 

Weapon  Outline  Symbols 

Eg 

1.9 

2.2 

2.2 

1.9 

1.9 

1.9 

1.9 

2.1 

1.9 

Master  Arm  Status  Indicator 

2.4 

2.3 

2.S 

2.5 

2.3 

2.2 

2.2 

2.2 

2.2 

E9 

Weapon  Status  Coding 

2.1 

2.1 

2.3 

2.3 

2.1 

1.9 

1.9 

1.9 

1.9 

1.9 

Weapon  Type/Number  Readouts 

1.9 

1.9 

Eg 

1.8 

1.6 

1.7 

1.7 

1.7 

1.9 

Display:  Stores  Programming 


Options  Available 

1.9 

2.1 

2.1 

1.9 

1.9 

1.9 

1.8 

1.8 

1.9 

1.4 

Indicator  Box 

1.4 

1.4 

1.5 

1.6 

1.5 

1.5 

1.5 

1.4 

1.4 

1.4 

SSS0S2-F/30>4 
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PILOTS'  RESPONSES  TO 


INFORMATION  INTERPRETATION  QUESTIONS 

This  part  of  ths  questionnaire  is  designed  to  elicit  x cu i 
opinions  on  ths  rslstivs  usability  of  ths  two  display  i.odss  - 
aonochrons  and  color.  Placs  a  numbs r  in  sach  column  t*  indicate, 
for  ths  monochrome  and  color  versions,  ths  degree  to  which  it  is 
easy  or  difficult  to  understand  the  particular  display 
Information. 


Easy  Difficult  _ _ 

Very  Moderately  Slightly  Neutral  Slightly  Moderately  Very 
1  2  3  4  5  6  7 


HUP  Monochrome 

Row  easy  or  difficult  is  it  to  interpret 
tha  following  information  on  HUD? 


Pathway 
Pitch  Ladder 

Threat  Alert  and  Summary  Information 
DBP  RLE  Arrow 
ATT  RLE  Arrow 

Vertical  Velocity  Indicator 
Ownship’s  Relationship  to  Terrain 
Waypoint  Locations 

Weapon  Release  Cue 


3.4 

T77 

T75 

T7* 


“27* 

“575 

“175 

“57* 


PSP 


How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  PSP? 


Planned  Plight  Route  3.3 
Surface-to-air  Threat  Type  (SAM, AAA)  2.? 
Surface-to-air  Threat  Mode 

(search,  track,  launch)  2.9 
Airborne  Threat  Type  (enemy,  unknown)  3.6 
Airborne  Threat  Mode  (search,  track, 

launch)  2.8 
Area  of  Airborne  Threat  Radar 

Coverage  Area  2.6 
Terrain  4 . 3 
Location  of  Waypoints  2 . 1 
Range,  Closing  rate  and 

Relative  Altitude  Readouts  3 . 3 


Color 


3.1 


2.4 

~TT5 

1.6 

“276 

2.2 

2.0 

“575 

TTf 


3.4 
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my  Difficult _ 

Very  Moderately  Slightly  Neutral  Slightly  Moderately 
1  2  3  4  5  6 


hsf 


Monochrome  Color 


Bow  easy  or  difficult  is  it  to  interpret 
the  following  information  on  HSF? 


Flight  Route 

Display  Range 

Fuel  Range  Rings 

Ownship's  Proxiaity  to  Mountains 

Surface-to-Air  Threat  Type 

(SAM, AAA) 

Surface-to-Air  Threat  Mode 
(prebriefed,  search,  track 
launch ) 

Airborne  Threat  Type  (enemy,  unknown 
Airborne  Threat  Mode  (prebriefed, 
search,  track,  launch) 

Ownship's  Position  Relative  to 
Enemy  Radar  Coverage 
Airborne  Target  Position  Relative 
to  Ownship  Radar  Coverage 
Ownship  NLE  Boundary  Arcs 
Enemy  MLE  Boundary  Arcs 


2.0 

171 

175 

ITT 

1.7 

T1 

171 

2.6 

1.6 

2.4 

171 

1.6 

*17? 

2.8 

2.3 

* 

1.8 

1.5 

1.9 

171 

ITS 

1.6 

171 

171 

Close  Look  Tormats  (CLF's) 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  Close  Look 
Formats  (Detail  and  Formation  versions)? 


Aircraft  Classification  (enemy, 

unknown,  friendly)  2.8 


Classification  Status  (probable 

or  known)  2.8 
Aircraft  Flight  Vector  3 . 0 
System  Recommended  Target  Coding  4 . 0 
Weapon  Assignment/Readiness  Coding  3.6 


Formation  CLF  only: 

Aircraft  Position  and  Movement  2 . 3 

Location  of  Ownship  Relative 

to  Formation  3.3 


1.7 


1.8 

"27? 

17! 

ITT 


2.1 


3.3 


Very 

7 
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my  Difficult  _ 

Wry  Moderately  Slightly  Neutral  slightly  Nodarataly  Wry 


12  3  4 

Close  Look  Formats  (Cont'd) 

5  6 

nonochrome 

Color 

Detail  CLF  only! 

Aircraft  Type  (fighter,  bomber. 

specific  type) 

1.6 

1.7 

Aircraft  Relative  Altitude 

“575 

Aircraft  Speed 

T7T 

2 , 2 

Engine  Status  Format 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the 
Engine  Status  Format? 


Commanded  Thrust,  Throttle 


Position  and  Actual  Thrust 

2.7 

2.3 

Fuel  Flow 

177 

2 . 8 

Oil  Pressure 

~m 

ITT 

Oil  Quantity 

177 

179 

B6T 

17? 

T 71) 

Damage/Failure  Coding 

n 

2.6 

Fuel  Status  Format 


How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the  Fuel 
Status  Format? 

Fuel  Quantity  1.8 

Pump  Status  (normal/failed)  5 . 5 

Damage/Failure  Coding  3.8 


1.7 

T75 

~2T5 


Easy _ _ _ _  Difficult  _ _ 

Very  Moderately  Slightly  Neutral  slightly  Moderately  Very 
1  2  3  4  5  6  7 


Electrical  Status  Format  Monochrome  Color 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  Electrical 
Status  Format? 


Element  Symbology  2 . 0 

Damage/Failure  Coding  2 . 7 

Hydraulic  Status  Format 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the  Hydraulic 
Status  Format? 

Aircraft  Element  Supported  3.1 

Level  of  Element  Support  {full 
support , single  threat  support, 
full  failure)  4 . 4 

Damage/Failure  Coding  4.1 

Stores  Status  Format 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the  Stores 
Status  format? 


Weapon  Complement  1 . 3 
Type  and  Number  of  Weapons  Selected  1 . 5 
Master  Arm  Status  18 
Selected  Weapon  Status  {assigned/ 

ready)  2.1 


1.6 

175 


2.4 


3.2 

2.6 


1.4 

T7J 

T7S 


1.7 


Easy  _ _ _  Difficult  _ 

Vary  Moderately  Slightly  Neutral  slightly  Moderately 
1  2  3  4  5  6 


Passive  Sensor  System  Monochrome  Color 

Status  Format 


Row  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the  Passive 
Sensor  System  Status  format? 

Status  of  Passive  Sensor  System  3.6  2 .6 
Area  of  Degraded  Coverage  “ 5  6  3 . 3 


Countermeasures  Status 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the 
Countermeasures  Status  format? 


Status  of  Jammer  (off, standby , on)  1.4 

Quantity  of  Expendable 
Countermeasures  Selected  2 . 0 

Permission  Required  Indicator  1 .6 

Chaff  Symbology  vs  Flare  Symbology  26 


.1 . 5 


1.5 

175 

175 


Very 

7 


022 


PILOTS'  RESPONSES  TO 
INFORMATION  CROSSCHECK  QUESTIONS 

This  part  of  the  questionnaire  is  designed  to  elicit  your 
opinions  on  the  relative  ease  or  difficulty  of  crosschecking 
information  across  displays  in  the  two  modes  -  monochrome  and 
color  *  Place  a  number  in  each  column  to  indicate,  for  the 
monochrome  and  color  versions,  the  degree  to  which  it  is  easy  or 
difficult  to  understand  the  information  across  displays. 


Easy _ _ _ _ _  Difficult  _ _ 

Very  Moderately  slightly  Neutral  Slightly  Moderately  Very 
1  2  3  4  5  6  7 

HUP,  PSF ,  and  HSF  Mdnochrome  Color 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  HUD, 

PSF  and  HSF? 

Flight  Path  Information  3 . 3  2.6 

Ownship' s  Position  Relative 

to  Terrain  5.5  4.2 

Threat  or  Missile  Position 

Relative  to  ownship  2.9  2 . 4 

HUD  end  HSF 

How  easy  or  difficult  is  it  to  crosscheck 
the  fol_owing  information  on  the  HUD  and 
HSF? 

Ownship  ATT  MLE  Arrow  and  MLE 

Boundary  Arcs  3 . 5  2.6 

Enemy  DEF  MLE  Arrow  and  MLE 

Boundary  Arcs  3.7  2 . 7 
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Difficult 


Very  Moderately  Slightly  Neutral  sTTgRtly  Moderately  Very 
1  2  3  4  5  6  7 


FSF  and  HSF 


Monochrome  Color 


How  easy  or  difficult  is  it  to  crosscheck 
the  following  information  on  the  PSF  and 
HSF? 

Surface-to-Air  Threat  Location 
Surface-to-Air  Threat  Type  (SAM, AAA) 
Surface-to-Air  Threat  Mode  (search, 
track,  launch) 

Airborne  Threat  Location 
Airborne  Threat  Type  (enemy,  unknown) 
Airborne  Threat  Mode  (search,  track, 
launch) 


2.5 
TjS 
"I 7* 
“37T 


1.8 

TJ. 

1.6 

~T7l 

T75 


CLF  and  HSF 

How  easy  or  difficult  is  it  tc  crosscheck 
the  following  information  on  the  CLF  ana  HSF? 


Aircraft  Type  (enemy,  unknown, 
friendly) 

Aircraft  Heading 
Target/Weapon  Status 

CLF  and  Stores  Status 


275 

“275 


1.8 

“275 

“175 


How  easy  or  difficult  is  it  to  crosscheck 
the  following  information  on  the  CLF  and 
Stores  Status  format? 


Target/Weapon  Status 


WSO  RESPONSES  TO 


LOW  LEVEL  PENETRATION  SEGMENT  QUESTIONS 

The  purpose  of  this  questionnaire  is  tc  elicit  your  opinions  of 
the  symbology  used  in  this  6tudy.  Defined  below  are  the 
attributes  or  characteristics  that  you  will  rate  for  each  type  of 
display  symbology. 

Usability  in  Color.  How  easy  was  it  to  make  use  of  this 
display  element  in  the  color  display  mode? 

Conspicuousness  in  Color.  How  easy  was  it  to  see  this 
display  element  in  the  color  display  modo? 

Usability  in  Monochrome.  How  easy  was  it  to  make  use  of 
this  display  element  in  the  monochrome  display  mode? 

Conspicuousness  in  Monochrome.  How  easy  was  it  to  see  this 
display  element  in  the  monochrome  display  mode? 

Location.  Is  this  format  element  in  the  right  place  and  on 
the  right  display? 

Meaning.  How  clear  or  obvious  is  the  meaning  of  this  format 
element? 

Precision.  Does  this  format  element  convey  its  information 
with  the  appropriate  level  of  precision? 

Timeliness.  Is  this  format  element  available  to  you  at  the 
right  time  and  for  the  right  duration? 

Training.  How  easily  could  this  format  element  be  learned? 

Workload.  Does  this  format  element  contribute  to  workload 
or  relieve  it? 


The  symbology  element  is: 


1. 

2. 

3. 

4. 

5. 

6. 
7. 


Very  Goo< 
Moderate 
Slightly  G 
Neutral 
Slightly  P< 
Moderate 
Very  Poor 


Display:  PSF  GROUND  MODE 

Display  Element: _ 

Ownship  Symbol _ 

Planned  Flight  Route 
Ground  Grid 
Terrain  Altitude  Coding 
Airspeed  Readout 
Heading  Readout 
Altitude  Readout 
Ground  Threats: 

Threat  Lethality  Volume 
Track  Mode  Tractor  Beam 
Launch  Mode  Tractor  Beam 

Missile  Symbol 

Lock  On  Circle  (around  ownship) 

Airborne  Threat  Sy  mbol 
New  View  Readouts 

Ownship  Preview  Symbol 
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with  respect  to  this  attribute 


5or 

ly  Poor 


In 

In 

Mono¬ 

Color 

chrome 

Mode-independent 

Attributes 


2.3 

0 

3.1 

3.8 

B 

2.1 

2.5 

B 

B 

L _ 

2.6 

2.8 

m 

3.6 

B 

B 

2.3 

3.1 

2  8 

2.3 

28 

2.9 

2.6 

3.1 

B 

B 

2  9 

2  6 

2.6 

26 

3.1 

2.5 

b 

B 

B 

2.3 

B 

3.6 

2.3 

2  3 

3.1 

2.2 

1.9 

B 

2  2 

B 

19 

1  7 

1.6 

1.9 

2  3 

2.6 

1-9 

2.8 

2  1 

2  1 

2  2 

17 

1.8 

2.3 

28 

2.5 

2.5 

2.8 

2  8 

2  9 

2.1 

1.7 

18 

B 

3.1 

1.9 

1.5 

4.1 

B 

2.3 

2.3 

2.8 

2.1 

1.9 

2.3 

1.9 

2.1 

3.1 

3.3 

26 

1.8 

1.9 

1.8 

18 

1.9 

2.5 

2.6 

3.1 

3  5 

B 

2  3 

2.0 

19 

B 

2.2 

2.3 

B 

2.9 

3  1 

2.3 

2.1 

2.3 

2.2 

B 

2  6 

3  3 

3  8 

2  8 

2.6 

2.9 

2.6 

2  6 

3.2 

2.8 

28 

3.3 

3.5 

28 

B 

B 

28 

2  8 

B 

2.9 

2  6 

3  3 

3  1 

B 

2  8 

25 

2  5 

2  6 

28 

3  5 

B 

B 

eg 

B 

3  1 

B 

3.2 

3  1 

B 
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1 .  Very  Good 

2.  Moderately  Good 
J  3.  Slightly  Good 

The  symbology  element  is:  <4.  Neutral 

'5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


Display:  KSF  GROUND  MODE 


In 

In 

Mono- 

Color 

chrome 

Mode-independent 

Attributes 


Ownship  symbol 

1.5 

1.6 

B 

B 

2.1 

1.5 

2.3 

1.9 

17 

1.8 

Planned  Flight  Route 

1.6 

1.4 

2.1 

2.5 

1.9 

1.5 

1.6 

1.6 

1.7 

1.8 

Flight  Route  Alternates 

2.2 

1.9 

B 

3  2 

2.3 

2.3 

2.0 

3  8 

29 

3.3 

Display  Range  Rings 

3.2 

2.5 

3  5 

2  9 

2  9 

3.5 

3.7 

2  6 

3.0 

3  7 

Fuel  Range  Rings 

4.1 

3.3 

4.1 

3  6 

3  1 

43 

36 

3.5 

3  5 

39 

Terrain  Above  Altitude 

26 

2.3 

3.8 

3  8 

2.3 

1.9 

2.0 

1  9 

2  7 

26 

Ground  Threats: 


AAA  Lethality  Area 

1.5 

B 

B 

1.5 

1.5 

1.7 

1.5 

1.5 

1.6 

SAM  Lethality  Area 

1.5 

1.3 

B 

B 

1.5 

1.6 

1.7 

1.5 

1.5 

1  6 

Track  Mode  Tractor  Beam 

1.5 

1.9 

2.1 

2.9 

1.8 

1.7 

1.5 

14 

1  6 

17 

Launch  Mode  Tractor  Beam 

2.2 

2.5 

B 

3  2 

2.2 

2.4 

2  1 

2  1 

2.1 

2  2 

Missile  Location  and  Type 

2.3 

2.8 

2.9 

B 

2.6 

1  9 

2  4 

2.0 

2  1 

2.4 

Lock  On  Circle 

3.0 

B 

B 

m 

26 

2.5 

2  7 

2  1 

2  3 

2.6 

Airborne  Threat  Symbols 

28 

28 

3.3 

3.3 

3  0 

2.7 

2  3 

2  3 

2  3 

2.3 

Ownship  Missile  Symbol 

2  6 

B 

3  2 

3  8 

2  8 

2  5 

28 

2  6 

28 

2  8 

FLOT 

26 

2  2 

28 

2  3 

2  1 

2  1 

2  4 

2  4 

3  1 

General  Cursor  Symbol 

2.1 

.. 

2  3 

B 

2  0 

1  6 

2.3 

2  1 

2  1 

2  3 

Waypoints  Target  Symbol 

1.7 

1  9 

_ 

2  5 

:u 

:!j 

2  2 

18 

2  1 

1.8 

Display  Range  Readout 

2  ? 

fl 

2  7 

2  1 

_ 

i  9 

2.1 

24 

2  5 

SS5052-O/308-6 
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The  symbology  element  Is: 


1  Very  uood 

2.  Moderately  Good 

3.  Slightly  Good 

4.  N*utr«l 

5.  Sfrjhtly  Poor 

■6.  Moderately  Poor 
7.  Very  Poor 


Display:  Countermeasures  Status 


Jammer  Status 


Expendables  Symbology 


Expendables  Status 


/  / 


with  respect  to  this  attribute 


In 

In  Mono- 
Color  chrome 


Mode-i  ndependent 
Attributes 


2  1  1.7  1 2.4  2.7  1.9 


1.3 

1.3 

1.5 

1.6 

1.6 

1.6 

1.9 

1.9 

1.9 

1.5 

1.9 

2.1 

Display:  Countermeasures 
_ Programming _ 


Quantity  Selection  Options 

1.8 

1.5 

1.9 

1.6 

1.8 

1.6 

2  1 

1.8 

1.6 

1.8 

Release  Method  Options  (single, 
burst,  salvo) 

1.9 

1.7 

M 

1.8 

1.8 

17 

2.1 

1.7 

1.9 

1.9 

Permission  Required  Selection 

2.2 

1.8 

2.3 

1.9 

2.3 

1.5 

1.5 

2.3 

SSS052-P/03f > 
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WSO  RESPONSES  TO 
AIR-TO-AIR  BVR  SEGMENT  QUESTIONS 

The  purpose  of  this  questionnaire  is  to  elicit  your  opinionit  of 
the  symbology  used  in  this  study.  Defined  below  are  the 
attributes  or  characteristics  that  you  will  rate  for  each  type  of 
display  syabology. 

Usability  in  Color.  How  easy  was  it  to  make  use  of  this 
display  element  in  the  color  display  mode? 

Conspicuousness  in  Color.  How  easy  was  it  to  see  this 
display  element  in  the  color  display  mode? 

Usability  in  Monochrome.  How  easy  was  it  to  make  use  of 
this  display  element  in  the  monochrome  display  mode? 

Conspicuousness  in  Monochrome.  How  easy  was  it  to  see  this 
display  element  in  the  monochrome  display  mode? 

Location.  Is  this  format  element  in  the  right  place  and  on 
the  right  display? 

Meaning.  How  clear  or  obvious  is  the  meaning  of  this  format 
element? 

Precision.  Does  this  format  element  convey  its  information 
with  the  appropriate  level  of  precision? 

Timeliness.  Is  this  format  element  available  to  you  at  the 
right  time  end  for  the  right  duration? 

Training.  How  easily  could  this  format  element  be  learne*? 

Workload.  Does  this  format  element  contribute  to  workload 
or  relieve  it? 
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Tht  symbology  tlamant  is: 


1.  Vary  Good 

2.  Modarataly  Good 

3.  Slightly  Good 

4.  Neutral 

5  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


with  respect  to  this  attribute 


In 

in 

Mono- 

Color 

chroma 

Mode-independent 

Attributes 


Display:  PSF  AIR  MODE 
Display  Element: 


Ownship  Symbol 


Planned  Flight  Route 


Terrain  Altitude  Coding 


Ground  Grid 


Groundpoint  Symbol 


Mach  Readout 


Heading  Readout 


Altitude  Readout 


Airborne  Threats: 


Threat  Symbols 


Radar  Coverage  Sectors 


Track  Mode  Tractor  Beam 


launch  Mode  Tractor  Beam 


Missile  Symbol 


Lock  On  Circle 


SAM  Missile  Symbol 


AAA  Gun  Symbol 


T racked  A/C  Readouts  range 


closing  rate 


relative  altitude 


Ownship  Preview  Symbol 


2.6  2.3  3.1  3  3  2  0  1.8  2.1  1.6  1  6  2.1 


2.3  2.2  2.7  2.7  2.1  2 


2.8  |  2.1  4.3  4.2  <2  1  j 2  5  1 3  3  |2.5  |2.6  |3.1 


3.4  2.3  1 3.9  3.2  2.1  2.3  2.2  2.0  2  1  2  8 


mmmm 

BOOB 


IQ 

a 


3  8  4.7  5.0  3.3  3.6  3  5  3  3  3  8  4  1 


2.7  2.4  2  7  2.5  12.5  2  1  2  1  2.1  24  2 


3.1  2.1  13.2  2.3  1  8  2.4  1  8  1  8  2  3  3  1 


2.2  1.9  2.2  2.0  2  3  2  0  1.5  2  5  1.9  2  5 


bdooboUqs 


1.3  2.5  3.1  1.5  1.9  1.6  1.7  1.6  2  1 


1.9  |  2.2  1 2  2  j  2.9  1 1  8  2.1  1.9  2  1  1.7  2  6 


2.3  2.3  2  6  3  1  12  2  2  1  2  0  2  1  1  9  2  6 


90 

m 


2.4  2.4  12.9  3.4  1  8  1.7  2  0  2  0  16  2 


2.2  2.0  2  9  3  128  2  1  2  3  24  1  9  2 


iSSB 


9BBB 

BIBB 


2  3  1  8  2  5  1  9  1  7  26 


2.3  24  30  3  4  2.4  19  2  5  2  0  1  8  2  6 


1  9  2  5  2  2  2  8  30  2  3  1  9  2  3  2  4  2  9 


26 

29 

3  3 

2  2 

2  5 

29 

31363641 


29303140 
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The  symbology  element  is: 


V  V*ry  Good 
2.  Moderately  Good 
3  Slightly  Good 

4.  Neutral 

5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


Display:  HSF  AIR  MODE 
Display  Element: 


Ownship  Symbol 


Planned  Flight  Route 


Display  Range  Rings 


Fuel  Range  Rings 


AirborneThreats: 


Aircraft  Symbols 


Threat  Radar  Coverage  Sector 


Track  Mode  Tractor  Beam 


Launch  Mode  Tractor  Beam 


MLE  Boundary  Arcs 


Missile  Location  and  Type 


Ownship  Radar  Coverage  Area 


Ownship  Tractor  Beam 


MLE  Boundary  Arcs 


Ownship  Missile  Symbol 


SAM  Missile  Symbol 


AAA  Gun  Symbol 


Waypoint  Target  Symbols 


Display  Range  Readout 


with  respect  to  this  attribute 


in 

In 

Mono¬ 

Color 

chrome 

Mode-i  ndependent 
Attributes 


1.3  1 1.6  11.9 


1.8  2.0  12  ?  2.9  1 1.6  1.6  1.9  1.8  16  2.0 


I 


1.5 

1.3 

18 

1.6 

2  1 

3  1 

3.3  2.7  3.4  3.1  2.6  2.8  2  7  2  1  3 


3.9  5  0  4.0  3  9  4  2  4  7 


1.5  1.6  12.9  3  5  18  1  9 


1.8  1.5  1 2.5  3  1  1 1  7 


2  3  2  3 
19  20  22 
17  I  1 7  1 1 6  16  17  16 


1.5  1.7  2.2  29  1  7  1  7  1  6  1  6  1  7  1  6 

1.9  2.1  2.6  3  5  1  9  18  2.1  1.9  1  9  2.2 


2.5  2.6 


1.7 


2.6  2.6  2.1 


18  1.9  2  1  1  9  1  9  2  4 


1.5  1.3  2  3  2  6  1.9  1  8  1  7  1.7  1  8  1  9 

2.4  2  5  3  3  3  6  1.8  1  9  19  1  8  2.1  2  0 


3.2  3 


3.1  3  5  38  30  30  3  6 


2.6  2  4  3.1  3.3  2.3  2  2  2  5  2  5  1  9  2  7 

2  6  2  6  3  3  36  1  7  1.0  2  0  1  9  1  9  24 


3  5  3  9  1.7  1.9  ?0  19  2  1  2  6 

2  0  2  1  2  4  2  7  1  5  17  1  6  1  7  1  7  1.6 

2  7  2  3  2.1  2  1  2  3  19  2  0  2  7 
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1.  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  <  N*utral 
1  '5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 


>  with  respect  to  this  attribute 


In 

In 

Mono- 

Mode-independent 

Color 

chrrme 

Attributes 

Display:  CLF  DETAIL  Version 
Display  Element: 


Aircraft  Classification  Symbols 

1.5 

1.4 

2.5 

3.1 

1.9 

2.1 

1.6 

1.9 

2.3 

2.3 

Aircraft  Type  Alphanumerics 

1.4 

1.4 

1.7 

16 

17 

1.5 

1.4 

1.8 

2.0 

1.9 

Aircraft  Flight  Vectors 

m 

1.8 

2.6 

2.1 

ED 

2  3 

2.2 

23 

2  4 

2.1 

Aircraft  Relative  Altitude  Readouts 

2.8 

2.3 

m 

D 

2.9 

26 

2.2 

2  3 

D 

29 

Aircraft  Mach  Readouts 

2.3 

2.3 

2.6 

2.6 

m 

1  9 

1  5 

2  3 

26 

28 

Target  Recommendation  Coding 

2.1 

2.1 

3.8 

m 

1.9 

2  4 

20 

2  4 

2  7 

24 

W  eapon  Status  Coding 

1 

m 

m 

1.9 

2.1 

2.1 

1.7 

2.2 

22 

Aircraft  ID  Numbers 

i 

1 

2.3 

2.5 

2.3 

2.2 

1.6 

1.6 

3.1 

o 

SSS052-S/308-6 
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/*  *1 

1 .  Very  Good 

2.  Moderately  Good 

3.  Slightly  Good 

The  symbology  element  is:  \  f  Neutral 
'  5.  Slightly  Poor 

6.  Moderately  Poor 

7.  Very  Poor 

>  with  respect  to  this  attribute 

In 

Color 

In 

Mono-  Mode-independent 

chrome  Attributes 

Display:  CLF  FORMATION  Version 

Display  Element:  /  /cS 

' /*/ 
m 

v7 

w 

/ 

Aircraft  Classification  Symbols 

1.3 

1.3 

1.9 

0 

1  7 

15 

1.5 

1.7 

1.7 

18 

Aircraft  Flight  Vectors 

1.9 

1.8 

2.1 

Jn 

9 

2.1 

9 

2.0 

2  3 

2  3 

Target  Recommendation  Coding 

1.8 

2.1 

3  3 

5 

1  8 

19 

9 

3.1 

Weapon  Status  Coding 

1.8 

2.3 

2.9 

0 

1  9 

26 

1  8 

17 

9 

Aircraft  ID  Numbers 

2.1 

1.8 

o 

22 

2  1 

2  2 

1  8 

16 

2  6 

9 

Aircraft  Geometric  Arrangement 

1.4 

1.5 

1.6 

1  6 

1  7 

1  9 

2  1 

1.7 

2  3 

2.1 

C'vnship  Bearing  Vector 

2.5 

m 

28 

m 

23 

26 

9 

2  2 

26 

CLF  Range  Change  Feature 

m 

9 

4.1 

3.9 

29 

38 

3.1 

24 

3.1 

SS50S2-T/308-6 
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The  symbology  element  is: 


1. 

2. 

3 

4 

5. 

6. 

7. 


Very  Good 
Moderately  Good 
Slightly  Goiod 
Neutral 
Slightly  Poor 
Moderately  Poor 
Very  Poor 


>  with  respect  to  this  attri  bute 


Display:  Stores  Status 

In 

In  Mono-  Mode-independent 

Color  chrome  Attributes 

f 

Weapon  Selected  Coding 

1.4 

1.4 

2.1 

2.9 

1.6 

1.8 

1.5 

1.5 

1.5 

1.5 

Weapon  Outline  Symbols 

1.4 

1.5 

1.9 

2.6 

1.6 

1.8 

1.5 

1  5 

1.6 

1.5 

Master  Arm  Status  Indicator 

1.4 

1.6 

1.9 

26 

1.8 

1.5 

13 

1.3 

1.4 

15 

Weapon  Status  Coding 

1.8 

2.3 

2.9 

4.1 

2  1 

D 

m 

1.9 

2  2 

2.1 

Weapon  Type/Number  Readouts 

1.8 

1.8 

m 

2.2 

2  2 

1.9 

16 

1.7 

19 

1.9 

Display:  Stores  Programming 


Options  Available 

1.5 

1.4 

16 

D 

1.7 

1.5 

1.5 

1.6 

1.5 

1.6 

Indicator  Box 

1.4 

1.4 

1.5 

16 

1.6 

1.5 

1.4 

16 

1.5 

16 

SS50520-U/308-6 
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WSO  RESPONSES  TO 

INFORMATION  INTERPRETATION  QUESTIONS 

This  part  of  the  questionnaire  is  designed  to  elicit  your 
opinions  on  the  relative  usability  of  the  two  display  modes  - 
monochrome  and  color.  Place  a  number  in  each  column  to  indicate, 
for  the  monochrome  and  color  versions,  the  degree  to  which  it  is 
easy  or  difficult  to  understand  the  particular  display 
information. 


Easy _ _ _  Difficult  _ _ 

Very  Moderately  Slightly  Neutral  Slightly  Moderately  Very 
1  2  3  4  5  6  7 

HUD  Monochrome  Color 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  PSF? 


Planned  Flight  Route  3 . 5  3 . 0 

Surface-to-Air  Threat  Type 

(SAM, AAA)  3.6  1.4 

Surface-to-Air  Threat  Mode 

(search,  track,  launch)  3.5  2.0 

Airborne  Threat  Type  (enemy, 

unknown)  4.6  2 . 5 

Airborne  Threat  Mode  (search, 

track,  launch)  4.1  3.1 

Area  of  Airborne  Threat  Radar 

Coverage  Area  3.1  2.0 

Terrain  4.7  2.2 

Location  of  Waypoints  2.7  1.9 

Range,  Closing  Rate,  and 

Relative  Altitude  Readouts  2.6  2.4 
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Easy  _ _ _____ 

Very  Moderately  Slightly  Neutral 
12  3  4 


Difficult _ _ 

Slightlv  Moderately  Very 
5  “  6  7 


HSF  Monochrome  Color 

How  easy  or  difficult  is  it  to' interpret 
the  following  information  on  HSF? 


Flight  Route 
Display  Range 
Fuel  Range  Rings 

Ownship's  proximity  to  mountains 
Surface-to-Air  Threat  Type 
(SAM, AAA) 

Surf ace-tc-Ai r  Threat  Mode 
(prebriefed,  search,  track, 
launch) 

Airborne  Threat  Type  (enemy, unknown 
Airborne  Threat  Mode  (prsbriefed, 
search,  track,  launch)  J 
Ownship's  Position  Relative  to 
Enemy  Radar  Coverage  » 

Airborne  Target  Position  Relative 
to  Ownship  Radar  Coverage 
Ownship  MLE  Boundary  Arcs 
Enemy  MLE  Boundary  Arcs 

Close  Look  Formats  (CLF's) 


1.8  1.3 

179  “17 ff 

T7T  177 

TTI  177 


2.9  1.6 


2.9  1.7 

)  171  171 


3.5  2.0 


2.5  1.7 


2.1  1.5 

173  2.8 

TTe  ITT 


Monochrome  Color 


How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  Close  Look 
Formats  (Detail  and  Formation  versions)? 


Aircraft  Classification  (enemy, 

unknown,  friendly  3 . 3 

Classification  Status  (probable 

or  known)  3.3 

Aircraft  Flight  Vector  2.1 

System  Recommended  Target  Coding  3.6 

Weapon  Assignment/Readiness  4 .1 

Coding 

Formation  CLF  only: 

Aircraft  Position  and  Movement  2 ♦ 3 

Location  of  Ownship  Relative  to 

Formation  2.6 


1.4 


1.6 

H 

177 

177 


1.7 


2.1 
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Easy _ _  _ _  _  Difficult  _ 

Very  Moderately  Slightly  Neutral  Slightly  Moderately 
1  2  3  4  5  6 

Close  Look  Formats  (Cont'd)  Monochrome  Color 


Detail  CLP  only: 

Aircraft  Type  (fighter,  bomber, 

specific  type)  1.8 

Aircraft  Relative  Altitude  2 . 9 

Aircraft  Speed  19 

Fuel  Status  Format 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the  Fuel 
Status  Format? 

Fuel  Quantity  2.0 

Pump  status  (normal/failed)  3.1 

Damage/Failure  Coding  32 

Stores  Status  Format 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the  Stores 
Status  Format? 


Weapon  Complement  1.6 
Type  and  Number  of  Weapons  Selected  1.9 
Master  Arm  Status  2 , 3 
Selected  Weapon  Status  (assigned/ 

ready)  3.6 


Countermeasures  Status 

How  easy  or  difficult  is  it  to  interpret 
the  following  information  on  the 
Countermeasure  Status  format? 


Status  of  Jammer  (off,  standby,  on)  1 . 8 
Quantity  of  Expendable 

Countermeasures  Selected  2 . 1 
Permission  Required  Indicator  2 . 4 
Chaff  Symbology  vs  Flare  Symbology  1 . 9 


1.4 

"271 

T7T 


1.4 

175 

T75 


1.3 

ITT 

ITT 


1.5 


1.4 

1.8 

ITT 

176 


Very 

7 
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WSO  RESPONSES  TO 
INFORMATION  CROSSCHECK  QUESTIONS 

This  part  of  the  questionnaire  is  designed  to  elicit  your 
opinions  on  the  relative  easy  or  difficulty  of  crosschecking 
information  across  displays  in  the  two  modes  -  monochrome  and 
color.  Place  a  number  in  each  column  to  indicate,  for  the 
monochrome  and  color  versions,  the  degree  to  which  *t  is  easy  or 
difficult  to  understand  the  information  across  displays. 


Easy _ _ _  Difficult  _ _ 

Very  Moderately  Slightly  Neutral  slightly  Moderately  Very 
1  2  3  4  5  6  7 


[  PSF  and  HSF  Monochrome  Color 

~  - -  - 

How  easy  or  difficult  is  it  to  crosscheck 
[  the  following  information  on  PSF  and  HSF? 


Surface-to-Air  Threat  Location  3 . 4  1.9 

Surface-to-Air  Threat  Type  (SAM, 

AAA)  3.3  1.5 

Surface-to-Air  Threat  Mode  (search, 

track,  launch)  3.4  1.8 

Airborne  Threat  Location  3.2  2.5 

Airborne  Threat  Type  (enemy, 

unknown )  4.1  2.4 

Airborne  Threat  Mode  (search, 

track,  launch)  3.5  2.6 


CLF  and  HSF 

How  easy  or  difficult  is  it  to 
crosscheck  the  following  information 
on  the  CLF  and  HSF? 

Aircraft  Type  (enemy,  unknown, 
friendly) 

Aircraft  heading 
Target/Wenpon  Status 

CLF  and  Stores  Status 

How  easy  or  difficult  is  it  to 
crosscheck  the  following  information 
on  the  CLF  and  Stores  Status  format? 

Target/Weapon  Status  4 . 0  2 . 1 


3.6  2.3 

2.9  ~T7T 

3.9  ~m 
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APPENDIX  D 

AIRCREW  QUESTIONNAIRE 
OPEN  ENDED  ANSWERS 

As  the  last  exercise  in  opinion  data  collection,  the  aircrew 
members  were  given  a  list  of  open  ended  questions  and  a  tape 
recorder.  2  evious  studies  have  shown  this  to  be  an  effective 
vay  to  elicit  ideas  not  otherwise  available.  Transcripts  of  the 
tapes  from  the  individual  pilots  and  WSOs  are  summarized  in  this 
Appendix.  The  richness  of  the  raw  data  made  summarizing 
difficult  but  an  attempt  was  made  to  represent  all  of  the  format 
ideas  presented.  As  might  be  expected,  both  agreements  and 
disagreements  appear.  There  were  a  few  comments  which  dealt  more 
with  the  simulation  than  the  formats;  these  were  excluded  from 
the  summaries. 


Question  3U  What  is  your  opinion  of  the  Pathway-in-the-Sky 
(PfTS )  on  the  HUD?  Does  it  provide  sufficient  information  for 
flight  path  control  during  low  level  flight?  During  which  flight 
and  mission  phases  is  it  most  useful?  Are  there  any  flight  or 
mission  phases  when  it  is  not  useful? 


S  Pilots 
4  Pilots 
4  Pilots 

4  Pilots 

3  Pilots 
3  Pilots 
3  Pilots 
3  Pilots 
3  Pilots 
2  Pilots 
2  Pilots 
2  Pilots 
2  Pilots 
2  Pilots 

1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 


Difficult  to  follow  if  off  path. 

Generally  effective.  Meeds  improvement. 

Less  useful,  not  useful,  don't  need  in  air-to- 
air. 

The  pits.  Eliminate  pathway,  inadequate. 
Should  not  be  primary  aid. 

Good  cue. 

Better  in  low  level  than  at  altitude. 

Adequate  for  low  level. 

Too  sensitive. 

Completely  inadequate  as  a  steering  device. 
Useful  straight  and  level,  not  in  maneuver. 
Adequate  with  autopilot  on. 

Good  as  preview  of  coming  commanded  changes. 
Pitch  bad  during  level  flight. 

Transitional  flight  director  too  slow  and 
difficult  to  understand. 

Adequate  in  azimuth. 

Adequate  for  BVR  air-to-air. 

Confusing  in  color  and  monochrome. 

Path  easier  in  monochrome. 

Hard  to  tell  range  to  terrain  on  HUD. 


How  Change 


3  Pilots 

3  Pilots 
2  Pilots,  1  WSO 
2  Pilots,  1  WSO 
2  Pilots 

1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 

1  Pilot 
1  Pilot 


Need  steering  back  to  selected  point,  rather 
than  strict  return  path  to  some  arbitrary  point. 
Use  proportional,  ILS  type  steering. 

Pathway  should  provide  SAM  avoidance  guidance. 
Shoot  cue  should  be  more  noticable,  not  an  "X” . 
Tend  to  lose  gate.  Need  more  solid  steering 
target. 

Need  filled  path  in  air  mode. 

Make  pathway  selectable. 

Add  low  altitude  warning. 

Bank  index  should  continue  all  around  and  not 
flash  at  some  arbitrary  limit. 

In  air  mode,  don't  show  Mach  for  ownship  and 
knots  for  closing  velocity. 

Back  seater  needs  complete  HUD  information. 

VSI  indicator  should  peg  at  1500  or  2000  ft/min, 
with  digital  extending  beyond.  That  would 
provide  more  sensitivity  at  lower  vertical 
speeds . 
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Question  2.  What  is  your  opinion  of  the  Missile  Launch  Envelope 
TmlB)  information  presented  on  the  HUD  (MLE  arrows  and  carets)? 
Any  suggestions  for  changes? 


13  Pilots 
4  Pilots 
2  Pilots 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 


Very  useful  and  intuitive. 

Better  in  color. 

Good  in  color  or  monochrome. 

Do  not  need  color  coded  arrows. 

Make  shoot  cue  more  attention-getting. 
Incorporate  raw  range. 

Add  ranges  next  to  carets. 

Improve  target  and  threat  IDs  on  arrows. 

Have  carets  flash  when  within  no-escape  zone. 


Question  3.  What  is  your  opinion  of  the  Perspective  Situation 
Format  fPSF)?  How  well  does  it  provide  information  about  the 
tactical  situation?  Any  suggestions  for  changes? 


11  Pilots,,  5  WSOs 
9  Pilots,  5  WSOs 
3  Pilots,  6  WSOs 
3  Pilots,  2  WSOs 
3  Pilots 

2  Pilots,  2  WSOs 

2  Pilots 

1  Pilot,  1  WSO 
1  Pilot,  1  WSO 
1  Pilot 

1  Pilot 

2  WSOs 
1  WSO 
1  WSO 

How  Change 

8  Pilots,  6  WSOs 

1  Pilot,  1  WSO 
1  Pilot,  1  WSO 

1  Pilot,  1  WSO 
1  Pilot,  1  WSO 

1  Pilot 


Provides  good  information  well. 

Better  in  color. 

Good  threat  depiction. 

Particularly  useful  in  ground  mode. 

Difficult  to  use  for  terrain  clearance. 

Good  for  situation  awareness,  vertical 
situation,  tuning  HSr,  info. 

Heading,  airspeed  an:  'Icitude  redundant  with 
HUD. 

Useful  in  air  mode. 

Weak  in  air  mode. 

Good  idea.  Needs  work. 

Not  too  useful. 

Ground  grid  was  helpful. 

Confused  threats  and  terrain  in  monochrome. 
Better  than  we  have  row. 


Need  declutter  or  transparency  to  see  past 
near  threats  to  mountains  beyond. 

Ground  grid  not  helpful. 

Ownship  and  pathway  should  be  brighter  or 
larger  or  different  color. 

Make  PSF  into  a  flight  instrument. 

As  flight  instrument,  put  viewpoint  at 
ownship 

Add  shading  to  show  terrain  height  or  impact 
point. 
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tion  3.  (Conti nut d 


ruation  Format  (PSP 
About  the  tactical  situation? 


W 


What  ib  your  opinion  of  the  Perspective 
How  well  does  it  provide  infornation 
Any  suggestions  for  changes? 


1  Pilot 

1  Pilot 
1  Pilot 

1  Pilot 

1  Pilot 

2  WSOs 

1  WSO 
1  WSO 

1  WSO 
1  WSO 
1  WSO 
1  WSO 
1  WSO 

1  WSO 
1  WSO 

1  WSO 

1  WSO 

1  WSO 


Add  option  for  ground  or  airplane  stabilized 
choice. 

Change  PSF  range  with  HSF. 

Add  cockpit  outline  to  provide  visual 
reference. 

Add  vertical  grid  lines  for  attitude  and 
altitude  corrections. 

In  air  mode,  add  contrails  for  track  history. 
Add  selectable  ground  clearance  plane  and 
altitude  above  terrain. 

Code  terrain  ahead. 

Weed  stronger  indication  of  terrain  shape 
below  current  altitude. 

Add  aircraft  attitude  for  back  seat. 

Add  abbreviated  heading  tape  to  PSP. 

Wake  lock-on  circle  transparent. 

Add  selectable  in-cockpit  viewpoint. 

Show  true  rather  than  relative  altitude  for 
air  threats. 

Show  only  volumes  of  active  threats. 

As  flight  instrument,  add  artificial  horizon 
and  velocity  vector. 

Display  throughout  air  engage ment  without 
switching i 

For  air  targets,  show  aspect  angle,  heading, 
velocity,  range,  altitude  and  targetting. 

For  air  targets,  add  readouts  of  heading  and 
true  mach. 


Question  4 .  Did  you  use  the  "new  view"  feature  on  the  PSF  to 
change  the  viewpoint  of  the  format?  How  useful  is  the  feature? 


7  Pilots,  9  WSOs 
2  Pilots,  3  WSOs 
4  Pilots 

2  Pilots,  2  WSOs 

1  Pilot,  1  WSO 

2  Pilots,  3  WSOs 
1  Pilot,  1  WSO 

1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 


Used  it,  liked  it. 

Used  it,  didn't  like  it. 

Used  it  seldom  or  not  at  all. 

Used  it  in  ground  mode. 

Useful  in  air  mode. 

Best  looking  straight  forward. 

Best  rotated  15  degrees  down. 

Best  in  default  position. 

Didn't  like  extreme  adjustments. 

Good  for  flight  path  management,  bank  angle 
adjustment . 

Good  for  own  location  and  threat  avoidance. 
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Question  4 . ( Continued )  Did  you  use  the  "new  view"  feature  on  the 


change  the  viewpoint  of  the  fcraat? 
feature? 


How  useful  is  the 


1  Pilot 


1  Pilot 

2  WSOs 
1  WSO 
1  WSO 
1  WSO 


If  you  select  "new  view,"  change  viewpoint, 
deselect  "new  view,"  then  select  it  again, 
viewpoint  should  go  to  changed  value. 
Viewpoint  too  far  back. 

Useful  for  setting  personal  preference. 

Good  for  terrain  clearance. 

Useful  for  air-to-ground  transition. 

Hard  to  see  ownship  symbol  with  level 
viewpoint. 


Question  5.  What  is  your  opinion  of  the  Horizontal  Situation 
Format  (HSP)?  How  well  does  it  provide  information  about  the 
tactical  situation?  Any  suggestions  for  changes? 


9  Pilots,  13  WSOs 

4  Pilots,  2  WSOs 
3  Pilots,  2  WSOs 
1  Pilot,  1  WSO 
1  Pilot 
1  Pilot 
1  Pilot 
1  WSO 
1  WSO 

1  WSO 
1  WSO 


Excellent.  Good  situational  awareness 
picture. 

Provides  tactical  information  very  well. 
Color  better  than  monochrome. 

Liked  ability  to  de-center  ownship. 

It's  a  pretty  radar  scope. 

Not  good  for  close-in  work. 

Too  far  from  HUD  for  cross-check. 

Difficult  to  judge  altitude  in  HSF. 

Almost  indispensible  for  threat  and  terrain 
avoidance. 

Not  great  for  tactical  information. 
Important  format.  Make  it  larger. 


How  Change 


4 

2 

1 

1 

2 

1 

1 

1 

1 


Pilots, 

3  WSOs 

Pilots, 

2  WSOS 

Pilot, 

1  WSO 

Pilot, 

WSOs 

1  WSO 

Pilot 

Pilot 

Pilot 

Pilot 

Add  range  up  and  range  down  buttons  vs.  range 
change  cycle. 

Label  range  rings  or  make  th*:a  constant 
radius  in  NM. 

Terrain  should  be  available  from  high 
altitude  to  plan  descent. 

Show  preplanned  alternate  route  ear.'  ier. 

Flash  new  threats  5  seconds  or  until 
acknowledged  by  WSO. 

Put  ownship  halfway  between  center  and 
bottom. 

Move  time  and  distance  readouts  to  HUD  or 
PSF.  Too  far  away  on  HSF. 

Add  time-to-go  to  next  waypoint  in  air  mode. 
Too  cluttered  with  All  Threats  selected. 
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flotation  5.  (Continued).  What  is  your  opinion  of  the  Horisontal 
Situation  Format  (HSF)?  How  well  does  it  provide  information 
about  the  tactical  situation?  Any  suggestions  for  changes? 


How  Change 

1  Pilot 

1  Pilot 

1  Pilot 

1  Pilot 
1  WSO 

1  WSO 
1  WSO 
1  WSO 
1  WSO 

1  WSO 

1  WSO 
1  WSO 

1  WSO 
1  WSO 
1  WSO 
1  WSO 

1  WSO 
1  WSO 
1  WSO 

1  Pilot,  1  WSO 


1  Pilot,  1  WSO 


1  WSO 
1  WSO 


Add  declutter  switch  to  eliminate  non- 
iamediate  threats. 

Reaove  lock-on  circle  around  ownship.  It 
covers  other  information. 

Add  aid  for  reaching  specific  time-over- 
target. 

Add  predictive  vector  to  nose  of  ownship. 

Need  better  indication  of  terrain  below 
current  altitude. 

Show  WSO  aircraft  attitude  on  HSF. 

Show  full  3-D  ownship  launch  envelope. 

Need  North-up  option. 

Add  compass  indication  around  periphery  of 
format. 

Add  offset  to  left  and  right  as  well  as 
bottom. 

Add  weapon  and  countermeasures  select  status. 
Have  range  readout  show  total  range  and  range 
of  rings,  e.g.,  160/40. 

Change  range  scales  to  multiples  of  5  NM. 
Increase  range  options. 

Make  range  rings  dimmer. 

Indicate  selected  weapons  on  selected 
targets. 

Indicate  which  are  pop-up  threats. 

Add  airspeed  indicator. 

Add  low  altitude  warning. 

Add  ability  to  set  up  route  deviations  with 
waypoints  to  provide  path  guidance  to  pilot 
for  flying  around  threats. 

In  air  mode,  show  individual  target  bearing, 
closing  velocity,  aspect  angle,  altitude 
differential,  missile  assigned,  and  single  or 
multiple . 

Eliminate  Close  Look  formats  and  window  in 
the  information  on  the  HSF. 

In  transition,  may  need  two  HSF's,  one  on 
small  scale  and  one  on  large  to  monitor  both 
ground  and  air  situations. 
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Question  6.  Should  the  HSF  background  change  color  as  a  function 
of  Master  Mode?  For  example ,  green  background  in  GND  Mode  and 
black  background  in  AIR  Mode.  How  would  this  information  be 
useful? 


12  Pilots,  11  WSOs  No.  Leave  it  as  it  is. 

3  Pilots,  3  WSOs  Yes.  Might  help. 

1  Pilot,  2  WSOs  Didn't  notice.  Don't  know. 

Question  7.  How  easy  is  it  to  correlate  threat  type,  position 
and  mode  Information  across  the  PSF  and  HSF?  How  useful  is  it  to 
have  threat  information  presented  on  two  displays  with  different 
viewpoints? 

9  Pilots,  8  WSOs 

2  Pilots,  3  WSOs 
2  WSOs 
1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 
1  WSO 

1  WSO 
1  WSO 
1  WSO 
1  WSO 


Good.  No  change. 

Better  in  color. 

Need  range  to  threats  on  PSF. 

Not  that  useful. 

Good  in  ground  mode,  weak  in  air  mode. 

Need  to  be  more  specific  to  support  tactical 
decisions. 

Flash  threat  envelope  for  launching  threats. 
Good  in  air  mode,  not  as  important  in  ground 
mode. 

Identify  missile  type  on  lethality  envelopes. 
Make  threats  transpai  t  on  PSF. 

Indifferent  to  different  viewpoints. 

Good  for  identifying  pop-up  threats. 


Question  8.  How  useful  was  the  preview  feature  on  the  HSF  and 
PSF?  During  which  mission  phases  is  it  most  useful?  Any 
suggestions  for  changes? 


10  Pilots,  5  WSOs 

1  Pilot,  5  WSOs 
4  WSOs 

2  Pilots,  1  WSO 

3  WSOs 

2  Pilots 
1  Pilot 
1  Pilot 
1  Pilot 

1  WSO 
1  WSO 
1  WSO 


Worthless.  No  time  to  use  it. 

Good  for  briefing,  ingress  and  egress. 
Distracting  or  disorienting. 

Good  in  both  air  and  ground  phases. 

Limited  use. 

Good  but  didn't  use  it  much. 

Good  for  back  seater. 

Must  run  quicker. 

Replace  with  capability  to  center  HSF  on 
cursor-selected  location. 

Bad  to  displace  real  time  display. 

Revert  to  real  time  if  threat  launches. 
Computer  should  show  primary  recommendations 
and  alternates. 
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Question  9.  Which  display  (hud,  PSF,  HSF)  did  you  find  most 
useful  for  throat  information?  Lease  useful?  Was  the 
distribution  of  threat  information  across  the  HUD,  PSP,  and  HSP 
appropriate?  If  not,  what  would  you  change  about  the 
distribution? 


Most  Useful 


1  Pilot,  9  WSOs 
6  Pilots 

2  Pilots,  3  WSOs 

3  Pilots,  1  WSO 

4  WSOs 

2  Pilots,  1  WSO 
2  Pilots 
1  Pilot 
1  Pilot 
1  Pilot 


1  Pilot 
Least  Useful 


HSF  most  useful. 

HUD  most  useful. 

PSP  and  HSP  both  good. 

All  good. 

Used  HSP  for  planning  and  advance  work,  then 
PSP  for  execution  and  close-up  work. 

PSP  most  useful. 

PSF  good  for  threat  avoidance. 

HUD  and  HSF  tied  for  most  useful. 

HUD,  then  PSP,  then  HSF. 

HUD  well  arranged.  Missile  time-of-f light 
was  useful  for  timing  expendables.  Missile 
and  tractor  beams  on  HSF  good. 

Threat  warning  and  mode  change  in  HUD  good. 


3  Pilots 
2  Pilots 


PSP  least  useful. 

hud  least  useful  for  threats. 


Distribution  of  Threat  Information 


4  Pilots,  5  WSOs  Distribution  of  threat  information  good. 


How  Change  Distribution 


2  Pilots 
2  Pilots 
1  Pilot 

1  Pilot 

1  WSO 


Switch  positions  of  PSP  and  HSF. 

Add  more  threat  information  to  HUD. 

For  pilot,  combine  information  and  eliminate 
HSP. 

Time-to-go  for  missile  impact  useless  to 
pilot.  WSO  may  use  it. 

Reduce  threat  depiction  range  on  PSF,  so 
distant  threats  don't  cover  closer  ones. 
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Question  10.  What  is  your  opinion  of  the  Detail  Close  Look 
format?  Does  it  provide  adequate  and  useful  raid  assessment 
information?  Any  suggestions  for  changes? 


9  Pilots ,  13  WSOs 
4  Pilots 

2  Pilots ,  2  WSOs 
2  Pilots 
2  WSOs 
1  Pilot 


Good.  No  change 
Unusable  and  confusing. 

Better  in  color.  Worse  in  monochrome. 
WSO  used  this.  Pilot  didn't. 

Difficult  to  learn. 

Pilot  may  not  have  time  to  use  it. 


How  Change. 

3  Pilots,  3  WSOs 
2  Pilots,  4  WSOs 
1  Pilot,  1  WSO 

1  Pilot,  1  WSO 

2  WSOs 

1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 
1  Pilot 

1  WSO 

1  WSO 
1  WSO 
1  WSO 


1  WSO 


Integrate  formation  and  detail  CLPs. 

Hard  to  correlate  groups  between  HSF  and  CLF . 
Make  CLF  a  separate  display.  Don't  displace 
PSF  air  mode. 

Display  actual  rather  than  relative  altitude. 
Have  digital  readouts  default  to  airspeed 
rather  than  relative  altitude. 

Prefer  Formation  Close  Look  Format. 

Rather  have  PSF. 

Not  sure  flight  vectors  are  useful. 

Speed  readouts  are  good. 

Close  Look  Format  should  default  to  formation 
when  first  selected. 

Use  letters  rather  than  symbolic  coding. 
Symbolic  coding  of  fighters  and  bombers  would 
be  better. 

Hake  symbols  smaller  to  make  room  for  more  of 
them. 

Assign  missiles  to  air  targets  as  in  F14A. 

Add  arrows  to  flight  vectors. 

Have  digital  readouts  default  to  previous 
selection  (airspeed  or  relative  altitude) 
when  Close  Look  is  reselected. 

Use  textures  rather  than  gray  shades  in 
monochrome . 


Question  11.  What  is  your  opinion  of  the  Formation  Close  Look 
Format?  Does  it  provide  adequate  and  useful  information  about 
the  target  formation?  How  useful  is  TRACK  selection  feature?  Any 
suggestions  for  changes? 


13  Pilots,  6  WSOs 
1  Pilot,  1  WSO 
1  Pilot 
1  Pilot 
1  Pilot 


Good. 

Confusing. 

Barely  adequate. 

WSO  used  this.  Pilot  didn't. 

Correlated  well  with  HSF  for  BVR  information. 


i 
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§uestlon  11.  (Continued)  What  is  your  opinion  of  the  Formation 
lose  took  Format?  Does  it  provide  adequate  and  useful 
information  about  the  target  formation?  How  useful  is  TRACK 
selection  feature?  Any  suggestions  for  changes? 


1  Pilot 

1  Pilot 

1  WSO 
1  WSO 
1  WSO 
1  WSO 


Easier  to  find  who's  targeted  than  on  Detail 
Close  Look  Format. 

Altitude  and  airspeed  readouts  good  for 
targeting. 

Didn't  use  it  too  much. 

Difficult  to  correlate  with  HSF. 

Hard  to  follow  if  groups  split  up. 

Used  Detail  Close  Look  Format  more  because  I 
could  only  see  the  ones  Z  was  tracking  on  the 
Formation  format. 


TRACK  Selection 


4  Pilots ,  4  WSOs 
1  Pilot,  2  WSOs 

1  Pilot 

How  Change 

5  Pilots,  2  WSOs 

2  Pilots 

1  Pilot 

1  Pilot 
1  Pilot 
1  Pilot 

1  WSO 

1  WSO 

1  WSO 

1  WSO 
1  WSO 

1  WSO 


TRACK  select  was  useful. 

Did  not  use  TRACK  select  often  or  at  all. 
TRACK  select  was  not  helpful. 


Add  ID,  range,  bearing,  aspect  angle,  closing 
velocity,  target  speed,  etc.,  to  Formation 
CLF  and  eliminate  Detail  CLF. 

Hake  it  3-D  so  you  can  view  it  from  different 
angles. 

Use  ID  letter  for  group  and  number  for 
aircraft  within  group. 

Move  to  HPD. 

Hake  ownship  vector  more  conspicuous. 
Formation  should  be  relative  to  ownship  -  an 
eyeball  picture  of  what  crew  is  looking  at. 
Make  symbols  smaller  to  accommodate  more  of 
them. 

Aid  auto  scale  change  to  keep  targets  in 
field  of  view. 

Need  more  range  available  when  formation 
dispurses. 

Display  target  aspect  very  clearly. 

When  reselected,  keep  previously  selected 
range . 

Keep  all  the  aircraft  in  the  formation  on  the 
scope  even  if  tney  were  separated  by  a  few 
miles. 


D-10 


Question  12.  What  it  your  opinion  of  the  Engine  Status  Fermat? 
Does  the  composite  thrust  bar  provide  adequate  and  useful 
information  to  set  and  monitor  thrust?  Any  suggestions  for 
changes? 

12  Pilots,  1  WSO  Liked  it. 

7  Pilots  Like  the  composite  thrust  idea. 

1  Pilot,  3  WSOs  WSO  needs  this,  too. 

2  Pilots  Better  in  color. 

1  Pilot  Nice  to  have. 

1  Pilot  Did  not  like  it. 


How  Change 

3  Pilots,  1  WSO 
3  Pilots 
2  Pilots 

2  Pilots 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 

1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 

1  Pilot 
1  Pilot 


1  Pilot 
1  Pilot 


Show  oil  and  EGT  by  exception. 

Make  fuel  flow  digital,  take  out  arrows. 

Show  digital  fuel  flow  in  hundreds  of  pounds 
per  hour. 

Put  thrust  levels  together. 

Too  much  critical  information  in  one  place. 
Seemed  confusing.  Simplify. 

Make  actual  thrust  wider. 

Use  conventional  thrust  percent  or  analog 
display. 

Delete  throttle  setting  thrust  bar,  keep 
thrust  command  carets. 

Make  thrust  limits  more  obvious. 

Move  power  required. 

Need  digital  oil. 

Put  oil  and  EGT  closer  to  horizontal  center 
of  format. 

Show  oil  pressure  as  a  bar  graph,  like  oil 
quantity  and  EGT. 

Add  EGT  readout  next  to  EGT  gauge. 

Make  composite  format  showing  thrust,  fuel 
quantity,  CM  and  missile  status,  then  call  up 
other  informaiton  when  needed. 

Use  vertical  tape  for  fuel  flow. 

Add  fuel  totalizer. 


Question  13.  What  is  your  opinion  of  the  Stores  Status  format? 
Does  it  provide  adequate  and  useful  information  to  monitor  the 
type,  number  and  status  of  stores  on  board?  Any  suggestions  for 
changes? 


12  Pilots,  15  WSOs 
3  Pilots,  8  WSOs 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 
1  WSO 


Good.  Useful. 

Color  better. 

Nice  to  have.  Didn't  «?dd  much. 

Concept  good.  Needs  a  little  improvement. 
Good  in  both  color  and  monochrome. 

Liked  halos  and  what's  selected. 

Difficult  to  represent  variety  of  ordinance 
we  have  today. 
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Question  13  (Continued)  What  is  your  opinion  of  the  Stores 
status  format?  Does  it  provide  adequate  and  useful  information 
to  monitor  the  type,  number  and  status  of  stores  on  board?  Any 
suggestions  for  changes? 


How  Change 


1  Pilot 
1  Pilot 

1  Pilot 
1  WSO 
1  WSO 
1  WSO 


Add  readout  of  weapon  being  launched. 
Coloring  sometimes  confusing.  Use  letters 
instead. 

Too  big.  Digital  better. 

Add  shoot  cue  for  WSO. 

Add  target  assignments. 

Put  weapon  assignment  with  target  on  HSP, 
then  Stores  Status  would  be  less  important. 


Question  14.  What  is  your  opinion  of  the  Stores  Programming 
Format?  How  useful  is  it  during  preflight  set  up?  Would  the 
programming  options  be  useful  in  flight?  Any  suggestions  for 
changes? 


12  Pilots,  12  WSOs 
6  Pilots,  8  WSOs 
5  Pilots,  5  WSOs 
2  Pilots,  1  WSO 
1  Pilot,  1  WSO 
1  Pilot 
1  Pilot 
1  WSO 


Good. 

Would  be  useful  in  flight. 

Good  for  preflight. 

OK  but  too  large. 

Too  many  button  presses. 

WSO  should  use  in  flight. 

Set  and  forget.  Not  useful  in  flight. 
Awkward.  Ripple  and  salvo  not  universal, 
quantities  need  to  be  more  precise. 


How  Change 

1  Pilot 

1  Pilot 
1  Pilot 

1  Pilot 

1  WSO 

1  WSO 
1  WSO 


Have  default  (leftmost)  weapon  selections 
already  made,  crew  use  only  infliqht. 

Use  disk  or  tape  to  preload  information. 
Reduce  number  of  steps.  Perhaps  make  it 
HOTAS . 

Use  keyboard  rather  than  stepping  through 
selections . 

Continuous  menu  around  periphery  would  reduce 
button  presses. 

Allow  for  multiple  designation  of  ordinance. 
Allow  pretargeting  of  weapons  to  reduce 
workload  during  critical  periods. 
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Question  IS.  What  la  your  opinion  of  the  Countermeasures  Status 
Format?  Does  it  provide  adequate  and  useful  information  to 
monitor  the  level  and  status  of  electronic  expendable 
countermeasures?  Any  suggestions  for  changes? 


12  Pilots,  IS  WSOs 
2  Pilots,  1  WSO 
1  Pilot 
1  Pilot 
1  Pilot 
1  WSO 


Good. 

Color  better. 

Important  information. 

No  difference  between  color  and  monochrome. 
Didn't  use  it.  "C"  and  "F"  were  OK. 

Wetter  in  monochrome.  Maybe  darker  green 
would  help  the  color  version. 


How  Change 

3  Pilots 
1  Pilot 

1  Pilot 
1  Pilot 

1  WSO 
1  WSO 

1  WSO 
1  WSO 


Use  digital  for  expendables  remaining. 

Use  advisory  light  for  jammer  and  countdown 
for  expendables. 

For  ECM,  indicate  loss*  of  capability. 

ECM  standby  and  chaff  or  flares  low  all 
amber.  It's  confusing. 

Could  be  smaller. 

"Permission  required"  should  be  shown  where 
crew  is  looking. 

Add  ability  to  stop  automatic  deployment. 
Move  cue  for  "Permiesion  Required"  closer  to 
primary  field  of  view. 


Question  16.  What  is  yov.r  opinion  of  the  Countermeasures 
Programming  Format?  How  useful  is  it  during  preflight  setup? 
Would  the  programming  options  be  useful  in  flight?  Any 
suggestions  for  changes? 


15  Pilots,  12  WSOs  Good. 

2  Pilots,  6  WSOs  Useful  in  flight. 

1  WSO  Awkward  to  access  due  to  menu  cycling. 


How  Change 

1  Pilot,  1  WSO 
1  Pilot 
1  Pilot 
1  Pilot 

1  WSO 

1  WSO 
1  WSO 


Need  more  variations. 

Use  keyboard  rather  than  stepping  through. 
Preprogram  on  tape.  Plug  in  to  airplane. 
"Permission  required”  optic n  should  be 
available  when  airplane  is  maneuvering. 

Too  long  head  down  in  flight.  Have  single 
switch  for  another  option. 

Get  away  from  menus. 

"Permission  required”  option  should  be 
available  full  time. 
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Question  17.  What  it  your  opinion  of  the  Electrical,  Hydraulic, 
fuel  Systea  Status,  and  Pattive  Sensor  formats?  Do  they  provide 
an  appropriate  level  of  information  about  systea  health  and 
systea  problems?  Any  suggestions  for  changes? 


6  Pilots,  9  WSOs 
S  Pilots,  4  VfSOs 
4  Pilots,  1  WSO 

1  Pilot,  1  WSO 

1  Pilot 
1  WSO 


1  WSO 


Color  better. 

Good. 

Status  formats  should  come  up  automatically 
with  Master  Caution. 

Make  them  all  stapler.  Display  only  required 
actions. 

Systea  displays  for  engineers,  not  pilots. 
Should  be  able  to  know  what  systems  are  lost 
without  having  to  refer  to  specific  system 
format. 

Nice  to  have  schematics  for  electrical, 
hydraulic  and  fuel. 


Checklists 


1  Pilot  Checklist  good. 

1  Pilot  Integrate  checklists  on  status  formats. 


Electrical 


5  Pilots,  2  WSOs' 
1  Pilot,  1  WSO 
1  Pilot 
1  Pilot 


Electrical  good. 

Better  in  color. 

OK,  but  unneeded. 

Add  alphanuaerics  to  clarify. 


Hydraulic 

3  Pilots 
2  Pilots 
1  Pilot 
1  Pilot 

1  Pilot 
1  Pilot 
1  Pilot 

1  WSO 
1  WSO 
1  WSO 


Hydraulic  somewhat  difficult. 

Hydraulic  good. 

Good  in  color.  More  difficult  in  monochrome. 
Cluttered  and  unneeded.  An  indicator  light 
is  sufficient. 

Add  alphanumeric  list  of  lost  systems. 

Add  alphanumeric  labels. 

Substitute  alphanumeric  advisory  for  color 
coding  of  single  thread  systems. 

Good  in  both  color  and  monochrome. 

Simple  and  clear. 

Cryptic,  needs  training. 
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stien  17.  (Continued 


iNhat  is  your  opinion  of  ths  Electrical, 
tatus,  and  Psssivs  Sensor  foraats?  Do 


provide  an  appropriate  level  of  inforaation  about  systea 
health  and  systea  probleas?  Any  suggestions  for  changes? 


Fuel 

4  Pilots,  2  tVSOs 
2  Pilots 

1  Pilot 
1  Pilot 

1  WSO 

1  WSO 


Fuel  good. 

Add  alphanuaerics  to  identify  systea 
coaponents. 

OK  but  too  auch.  Totalises  would  suffice. 
Showing  puaps  and  rerouting  fuel  lines  a 
little  confusing. 

Difficult  to  read.  Should  be  aore  clearly 
designated. 

Fuel  status  should  show  both  total  and 
useable  fuel. 


Passive  Sensor 


a  pilots,  7  wsos 
a  pilots,  7  wsos 
1  Pilot 
1  wso 


■ven  worse  in  aonochroae. 
Passive  Sensor  very  difficult. 
West  picture. 

OK. 


Question  IS.  What  is  your  general  opinion  of  pictorial  displays? 
mat  do  you  like  best  about  the  display  foraats  used  in  this 
siaulation?  What  do  you  like  least?  Would  you  like  to  have 
pictorial  foraats  in  a  fighter  of  the  future?  ror  which 
displays? 


12  Pilots,  16  WSOs 

3  Pilots,  4  WSOs 

4  Pilots 

1  Pilot,  1  WSO 
1  Pilot 


Good. 

Like  these  foraats.  All  useful  tactically. 
I  like  thea.  They  are  very  close  to  what 
should  be  there. 

Reduced  workload  with  these  foraats. 
increased  situational  awareness. 


Color 

2  Pilots,  3  WSOs  Color  auch  better. 

1  Pilot,  1  W80  Color  easier  initially,  aonochroae  could  be 

effective. 

Love  color  but  soaetiaes  overused  here. 


1  Pilot 


Qwtttlwi  It .  ( Continued )  What  la  your  general  opinion  of 
pictorial  iTipIiya?  What  do  you  Ilka  boat  about  tha  display 
formats  uaad  in  this  aiaulation?  what  do  you  Ilka  laaat?  Mould 
you  liko  to  have  pictorial  foraats  in  a  fighter  of  tha  future? 
Por  which  displays? 

Llkod  boat 


1  Pilot,  1  WBO 
1  Pilot 

1  Pilot 

1  Pilot 
1  Pilot 

1  Pilot 
1  Pilot 

1  Pilot 

1  WSO 

1  WSO 
1  WSO 

1  WSO 
1  WSO 


Liked  Laaat 


1  Pilot 
1  Pilot 


1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 

1  WSO 


1  WSO 
1  WSO 


Liked  tha  BSP  and  3-D  aapacta  of  tha  PSP. 

BUD,  pathway-in-the-sky,  and  Bngina  Statua 
vara  especially  easy  to  laarn. 

Likad  tha  RLE  arrows,  aapacially  in  color* 
invaluable  in  air-to-air  coabet. 

Likad  tha  PSP  beat;  it  was  wall  dona. 

PSP  and  BSP  era  really  good  in  correlating 
overall  mission. 

Threat  inforaation  excellent. 

Bngina  and  ayataa  health  foraata  ware  vary 
good. 

Aircraft  claaaificationa  on  Cloae  Look  Detail 
format  are  great  -  aaay  to  interpret. 

BSP  and  PSP  helped  a  lot  in  navigation,  and 
in  threat  and  terrain  avoidance. 

BSP  and  PSP  together  are  excellent  aide. 

Likad  tha  volumetric  indications  of  SAMa  and 
AAAa 

All  the  ground  mapping  diaplaya  were  nice. 
Liked  the  counteraeaaurea  aatup  and  weapone 
programming. 


Liked  tha  BUD  laaat. 

Liked  the  pathway  (on  the  BUD)  leasts  it  waa 
too  looae,  a loppy,  and  hard  to  follow.  Not 
enough  information  on  the  BUD  and  PSP  to  keep 
out  of  the  ground. 

Pathway-in-the-aky  ia  not  the  graataat 
an ewer;  ahould  only  be  an  aid. 

Liked  monochrome  PSP  laaat,  aapacially  in 
Ion}  range  air-to-air. 

Liked  Detail  Cloae  Look  format  least;  too 
complex . 

Terrain  clearance  was  questionable. 

Target  information  on  HSF  and  Stores  Status 
display  was  vary  difficult  to  interpret  in 
monochrome. 

Likad  Passive  Sensor  Health  format  least. 
Liked  redundant  displays  least.  It  is 
possible  to  have  identical  displays  up  on 
either  side,  especially  in  heat  of  battle; 
irritating. 
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Question  IS.  (Continued)  What  it  your  general  opinion  of 
pictorial  displays?  What  do  you  likt  bast  about  tha  display 
forisats  usad  in  this  simulation?  What  do  you  lika  laast?  Would 
you  lika  to  hava  pictorial  formats  in  a  fighter  of  tha  futura? 
For  which  displays? 


Hctorlal  formats  in  Fighters  of  tha  Future 


3  Pilots ,  3  WSOs 
1  Pilot 

1  W80 

1  WSO 
1  WSO 


1  Pilot 
1  Pilot 

1  Pilot 

1  Pilot 
1  WSO 

1  WSO 


Would  lika  to  hava  pictorial  formats  in 
fighter  of  tha  futura i  absolutely. 

Would  lika  to  hava  KUO,  pathvay-in-the-sky, 
PSF,  HSF  and  Engine  Status;  all  are 
excellent. 

Tha  only  problem  I  foresee  is  lack  of  3-0 
awareness  in  soma  cases,  working  with  2~D 
continuously. 

Would  cut  down  on  workload  in  both  seats. 
Would  be  vary  good  if  craws  ware  wall  trained 
to  use  than.  Would  be  great  for  visual  or 
marginal  VNC  conditions  as  a  crosscheck. 

Would  hava  a  hard  time  accepting  them  in 
total  IF*. 

would  lika  HSF,  it  is  invaluable.  Also  HUD. 
HSF  should  be  on  our  planes  right  now;  no 
reason  not  to.  Sana  for  electrical  system 
formats,  hydraulics,  and  things  lika  that. 
Would  lika  at  laast  tha  PSF  and  KSF;  tha 
overall  view  that  they  provide  is  definitely 
needed . 

Would  lika  HSF  with  fast  update  and  scrolling 
feature,  ownship-stabilised. 

Would  like  to  have  the  PSP  as  a  radar  warning 
display,  and  the  HSF  as  a  projective  map 
display. 

Would  definitely  like  to  have  in  future 
aircraft.  Would  like  PSF  for  low  level;  HSF 
for  route  choice  and  low  level;  all  NPD's  for 

system  analysis. 


General  Comments  and  Suggestions 


2  Pilots 

2  Pilots 
1  Pilot 

1  Pilot 
1  Pilot 
1  Pilot 


Use  alphanumerics  where  necessary  to  clarify 
what  the  pictures  are. 

Add  trend  information  to  heading  readout. 
Would  like  "growing  bar"  symbology  for 
altitude  to  give  trend  information. 

Hake  digital  readouts  on  all  displays  larger. 
Too  much  symbology  on  all  the  displays. 

Use  symbology  that  carries  over  from  format 
to  format. 
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flotation  IS. (Continued)  What  it  your  gtnortl  opinion  of 
pictorial  displays?  What  do  you  likt  btat  about  tha  display 
formats  uaad  in  this  simulation?  What  do  you  Ilka  laaat?  Would 
you  Ilka  to  have  pictorial  formats  in  a  fighter  of  tha  future? 
ror  which  displays? 


1  Pilot 

1  Pilot 

1  Pilot 

1  WSO 
1  WSO 

1  wso 
1  wso 


Tons  down  color;  too  bright  for  dia 
visibility  situations  or  night  usa. 

Incraasa  rasolution  of  displays*  aspacially 
primary  control  display. 

Don’t  ovarusa  plcturas  whan  analog  displays 
adaguata. 

Try  to  dacluttar  as  ouch  as  potsibla. 
Navigation  and  throat  dapiction  aspacially 
good. 

Naad  pictorial  foraats  for  air  engagements, 
but  not  lika  this. 

Naad  aanauvaring  information  for  air 
ancountar . 


HUD  Coaaants  and  Suggastlons 


4  Pilots 
2  Pilots 

2  Pilots 

1  Pilot 

1  Pilot 
1  Pilot 

1  Pilot 

1  Pilot 

1  Pilot 

1  Pilot 

1  Pilot 

1  Pilot 
1  Pilot 
1  Pilot 


Haka  all  digital  raadouts  on  HUD  largar. 

HUD  should  hava  pitch  indicator  to  indicate 
where  to  bring  airplane  to  gat  above  terrain. 
Eliminate  transitional  flight  director  on 
BUD;  usa  pathway  to  bring  you  back  to 
preplanned  route. 

Whan  I’m  off  tha  pathway.  I'd  like  a  gross 
steering  cue  on  BUD  that  tells  me  which 
direction  tha  pathway  is. 

Add  angle  of  attack  to  tha  bud. 

Must  hava  "low  altitude"  visual  warning  on 
BUD. 

Show  airspeed  in  knots  on  BUD,  even  in  air 
mode. 

Saro  pitch  line  needs  to  be  more  conspicuous 
on  BUD. 

Change  caution  and  warning  alerts  on  HUD;  one 
suggestion  is  to  make  only  ownship  symbol 
flash. 

Preferred  monochrome  for  the  BUD;  weapon 
release  cue  was  difficult  to  see  in  color. 
Heading,  altitude  and  airspeed  should  be 
analog  on  HUD. 

Use  heading  tape  on  HUD. 

Put  heading  tape  on  horizon  or  bottom  of  HUD. 
Show  acceleration  and  angle  of  attack  on  HUD. 
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(fraction  18.  (Continued)  What  It  your  general  opinion  of 

Sictoriai  displays?  Mat  do  you  lilts  boat  about  ths  display 
oraats  ussd  in  this  simulation?  What  do  you  like  least?  Would 
you  lilts  to  have  pictorial  formats  in  a  fighter  of  tha  future? 
Por  which  displays? 

Othar  Coaaants  and  Suggestions 


2  W*Os 
1  Pilot 

1  Pilot 

1  Pilot 

1  Pilot 


1  Pilot 
1  WSO 


Storas  foraats  bast* 

Add  flight  vactor  to  PSP  and  HSF  to  aid  in 
flight  path  control, 
tngina  Status  format  was  too  busy  for 
critical  itaas. 

Bliainata  (caution  and  warning)  borders  on 
displays)  thay  ara  distracting  and  unnaadad. 
Too  much  tiaa  spent  cycling  through 
hydraulic,  Electrical  and  fuel  Status 
foraats;  indicator  lights  and  fuel  totalisar 
would  suffice. 

Systea  health  foraats  too  big  and  too 
detailed. 

Add  gun  symbol  or  alphanuaerics  to  AAA 
envelopes  on  BSP. 


Question  13.  In  this  simulation,  we  demonstrated  tha  use  of  a 
raw  cursor  designation  functions.  What  did  you  think  of  these 
functions?  What  do  you  think  are  appropriate  cursor  designation 
functions  in  fighter  aircraft? 


6  Pilots 

2  Pilots,  3  WSO* 

1  Pilot,  2  WSOs 

2  Pilots 
2  WSOs 


Didn't  use  it. 

Functions  appropriate. 
Basy  to  use. 

Nice  to  have  for  pilot. 


Better  tor  WSO. 


Functions  were  standard,  easy  to  learn. 


>roprlate  Cursor  functions 


2  WSOs 

1  Pilot 


Use  cursor  to  insert  waypoints  for  return  to 
planned  route. 

Important  tor  targeting,  designating 
aircraft,  taking  a  closer  look;  for  ground 
targets,  and  ground  information  from  sensors, 
Very  important,  especially  in  dual  mission 
aircraft. 
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Question  19.  (Continued)  xn  this  simulation,  we  demonstrated  tho 
um  ok  a  c««  cursor  designation  functions.  what  did  you  think 
of  those  functions?  whst  do  you  think  sro  appropriate  cursor 
designation  functions  in  fighter  sir croft? 


% 


Appropristo  Cursor  functions 


1  Pilot 

1  Pilot 
1  Pilot 

1  WSO 
1  WSO 

1  wso 
1  wso 
1  wso 
1  wso 


Cursor  should  bo  svsilsblo  to  identify 
waypoints,  targets;  to  offset  waypoints;  to 
update  present  position;  to  re-center  display 
around  cursor;  to  display  tine-and-distance 
to  cursor-display  on  SUP  or  PSP. 

Use  to  designate  a  threat  for  more 
information. 

Could  be  used  on  NPDs  to  select  modes,  or 
used  for  switching  to  keep  hands  on  stick  and 
throttle. 

Xn  the  AS,  it  would  be  very  useful  to  use  a 
cursor  to  designate  a  target  quickly. 

Useful  to  designate  new  flight  path,  for 
example  to  avoid  threats. 

Use  cursor  for  more  thorough  update  of 
navigation  system. 

Use  cursor  to  point  out  features  to  other 
crew  members. 

Meed  range  and  bearing  to  any  cursor-selected 
point. 

xn  air  mode,  use  cursor  to  invoke  expand 
function,  do  missile  assignment  and  control 
data  callup. 


Other  Comments  or  Suggestions 


2  Pilots 
1  Pilot,  1  wso 
1  Pilot,  1  wso 
s  wsos 
1  Pilot 
1  Pilot 
1  Pilot 
1  Pilot 

1  Pilot 
1  Pilot 


1  Pilot 


Cursor  difficult  to  use. 

Hake  cursor  bigger. 

Heed  better  cursor  control  method. 

Wove  cursor  control  keys  to  the  left. 

Cursor  position  and  use  are  good. 

Cursor  nevds  to  move  taster. 

Cursor  difficult  to  see. 

Cursor  button  should  be  controlled  by  left 
finger. 

Hove  cursor  control  to  throttle. 

Hake  cursor  a  multiple  position  switch  so 
that  the  entire  process  can  be  done  with  one 
finger  in  a  few  movements. 

Works  fine  as  long  as  you  can  hook  (desired 
item).  I  don't  think  that  there's  enough 
precision  from  what  you  can  see  on  the  HSF. 


D-20 


Question  19.  (Continued)  Xn  this  siaulation*  wt  deaonstrated  the 
use  of  a  lew  cursor  designation  (unctions.  what  did  you  think  of 
tkiM  functions?  what  do  you  think  sro  appropriate  cursor 
designation  functions  in  fighter  aircraft? 


Other  Coaaenta  or  tuooestions 


1  Pilot 

1  Pilot 

1  WSO 
1  WSO 

1  WSO 

1  WSO 

1  WSO 
1  WSO 

1  WSO 

1  WSO 

1  WSO 


Would  have  to  be  aechanised  to  be  hands-on- 
stick-and-throttle  for  the  pilot  to  be  able 
to  use. 

Have  larger  sone  of  acceptance  for  cursor. 

Oet  close  then  it  juaips  to  nearest  thing. 

Too  many  button  switches  necessary. 

Awkward*  because  the  cursor  was  controlled 
with  the  right  hand*  and  the  left  hand  had  to 
cross  body  to  reach  switches. 

Put  all  cursor  controls  on  the  control  stick* 
or  in  one  place. 

Works  well.  Originates  on  ownship*  which  is 
where  it  should. 

Cursor  designation  took  too  long. 

Very  sensitive  takes  sons  practice  to 
control. 

If  you  aiss  a  designation,  you  have  to  start 
all  the  way  over;  that  takes  too  auch  tiae. 
Have  several  alternate  routes  preprograaaed 
and  selected  with  a  control-display  unit  or 
switch  instead  of  cursor. 

Preview  function  was  slow  and  didn't  offer 
auch  inforaation. 

"New  View”  function  was  a  little  difficult. 


1  WSO 


